Pipeline Defaults and Credentials

When you configure a Transformer for Snowflake pipeline, you specify the Snowflake URL for your account, the warehouse, and the role for the pipeline. The pipeline uses this information to access Snowflake data.

As a new user, when you specify these details in your first pipeline, they are stored in your StreamSets account as your Snowflake pipeline defaults. Snowflake pipeline defaults are added in new pipelines so you do not have to configure that information over and over again. You can change your Snowflake pipeline defaults at any time in your StreamSets account.

The user credentials required to connect to Snowflake are requested when needed. They are also securely stored in your StreamSets account for reuse.

Note: If your organization uses deployed Transformer for Snowflake engines, these details do not apply to you.

Default Role and Overrides

The role used to connect to Snowflake can help limit access to your Snowflake account. You can specify the Snowflake role used for a pipeline in the following ways:
Snowflake pipeline defaults
When you specify a role as a Snowflake pipeline default in your StreamSets account, then that role is used in all new pipelines by default.
You can update the role as needed in individual pipelines.
Pipeline properties
If you do not have a role defined as a Snowflake pipeline default, the pipeline uses the Public role, by default. You can change this by configuring the Role property on the General tab of pipeline properties.

Specifying Snowflake Credentials

If you are a new user who has not provided your Snowflake credentials, Transformer for Snowflake requests the Snowflake credentials when you create your first Transformer for Snowflake pipeline. The credentials are then validated and stored securely for reuse.

You can update your Snowflake credentials as needed in your StreamSets account. You can also update the credentials when you configure a pipeline.
Important: When you specify Snowflake credentials, they replace any previously-stored credentials and are used for all subsequent connections to Snowflake.
Note: If your organization uses deployed Transformer for Snowflake engines, these details do not apply to you.
  1. In the pipeline canvas, click More > Enter Snowflake Credentials.
  2. In the Snowflake Credentials dialog box, configure the following properties, and then click Save:
    Snowflake Credentials Property Description
    Snowflake URL Default Snowflake URL to use. For example:

    https://<yourcompany>.snowflakecomputing.com/
    Username Snowflake user name.
    Authentication Method Authentication method to use: password or private key.
    Password Password for the Snowflake account.

    Available when using password authentication.
    Private Key Private key for the Snowflake account. Enter a PKCS#1 or PKCS#8 private key and include the key delimiters.

    For example, when entering a PKCS#8 private key, include the -----BEGIN PRIVATE KEY----- and -----END PRIVATE KEY----- key delimiters.

    Available when using private key authentication.

Applying Snowflake Pipeline Defaults

If you have Snowflake pipeline defaults defined in your StreamSets account settings, they are only used in new pipelines by default. You can apply Snowflake pipeline defaults to existing pipelines on a case-by-case basis.

When you apply Snowflake pipeline defaults, the Snowflake pipeline defaults defined in your account are added to the pipelines, overriding the corresponding property configurations. No changes occur for properties that are not defined in your StreamSets account.

For example, say you define the Account URL and Warehouse Snowflake pipeline defaults. When you apply the Snowflake pipeline defaults to an existing pipeline, the definitions for the Snowflake URL and Warehouse properties in the pipeline are replaced with the Snowflake pipeline defaults. Other pipeline properties remain untouched.

You might apply Snowflake pipeline defaults to an existing pipeline if the pipeline was created before you defined or updated Snowflake pipeline defaults.
Note: If your organization uses deployed Transformer for Snowflake engines, these details do not apply to you.
  1. In the pipeline canvas, click More > Use Snowflake Defaults.
    A warning message indicates that applying Snowflake pipeline defaults overwrites existing pipeline properties.
  2. If this is acceptable, click OK.
    On the General tab, pipeline properties are replaced with the Snowflake pipeline defaults that are defined in your StreamSets account.