What is IBM StreamSets for Snowflake?
IBM StreamSets for Snowflake is a cloud-native platform for building, running, and monitoring data pipelines on Snowflake.
A pipeline describes the flow of data from origin to destination systems and defines how to process the data along the way. As a pipeline runs, you can view real-time statistics and error information about the data.
IBM StreamSets for Snowflake uses the following components to manage your pipelines:
- Control Hub
- Control Hub is a public cloud service that you access using a web browser. Use Control Hub to build, manage, and monitor your pipelines.
- Transformer for Snowflake
- Transformer for Snowflake is an engine that processes Snowflake data using Snowpark client libraries. Use the engine to perform complex processing in Snowflake without having to write SQL queries or templates. Most organizations use the Transformer for Snowflake engine hosted and managed by StreamSets. Based on the account agreement for your organization, you can deploy Transformer for Snowflake engines as you do other engine types.
Note: At times, this documentation uses "StreamSets" to refer to "IBM
StreamSets".
The following image provides a general overview of the IBM StreamSets for Snowflake components for organizations that use the default hosted engine:
The following image provides a general overview of the IBM StreamSets for Snowflake components for organizations that use deployed engines:
