Snowflake Pipeline Properties
When you configure a Snowflake pipeline, you specify the Snowflake URL for your account. You can optionally define the Snowflake warehouse, database, schema, and role for the pipeline.
The pipeline uses this information to access Snowflake data, by default. You can override the database and schema in individual stages as needed.
If you find that you are entering the same Snowflake details in each pipeline, you can simplify pipeline configuration by configuring Snowflake pipeline defaults in your StreamSets account.
The user credentials required to connect to Snowflake are requested when needed, and then securely stored with your account details. For more information, see Snowflake Credentials.
Default Role and Overrides
- Snowflake pipeline defaults
- When you specify a role as a Snowflake pipeline default in your StreamSets account, then that role is used in all new pipelines by default.
- Pipeline properties
- If you enter a specific role in the Role property on the General tab of pipeline properties, then that role is used to connect to Snowflake when you run the pipeline.
- Snowflake credentials
- If you specify a role as part of the Snowflake credentials in your StreamSets account, then that role is used when one is not defined in any of the locations above.
Applying Snowflake Pipeline Defaults to Existing Pipelines
When you apply Snowflake pipeline defaults, the Snowflake pipeline defaults defined in your account are added to the pipelines, overriding the corresponding property configurations. No changes occur for properties that are not defined in your StreamSets account.
For example, say you define the Account URL and Warehouse Snowflake pipeline defaults. When you apply the Snowflake pipeline defaults to an existing pipeline, the definitions for the Snowflake URL and Warehouse properties in the pipeline are replaced with the Snowflake pipeline defaults. Other pipeline properties remain untouched.
You might apply Snowflake pipeline defaults to an existing pipeline if the pipeline was created before you defined or updated Snowflake pipeline defaults.
-
In the pipeline canvas, click
.A warning message indicates that applying Snowflake pipeline defaults overwrites existing pipeline properties.
-
If this is acceptable, click OK.
On the General tab, pipeline properties are replaced with the Snowflake pipeline defaults that are defined in your StreamSets account.