Snowflake Lookup

The Snowflake Lookup processor performs a lookup on a Snowflake table. The processor can return the first matching row, all matching rows, a count of matching rows, or a boolean value that indicates whether a match was found. When reading data from Snowflake, the processor stages the data in an internal stage.

When you configure the Snowflake Lookup processor, you define the Snowflake region, database, table, and schema to use. You specify the user account and password to use. You can optionally specify a custom role that overrides the default role for the user account. The user account or the custom role must have the required Snowflake privileges.

You can also use a connectionconnection to configure the processor.

You configure the record field to use and the table column to match against. You also specify the operator to use. You select the information to return, then configure related properties.

When returning one or more records, you specify the columns to return and optionally define a prefix for the resulting field names to prevent adding duplicate fields to the record. You can specify columns to sort by and the sort order. When returning multiple rows, you can specify a maximum number of rows to return.

When returning a count or boolean value, you define a name for the field to contain the results. If the field does not exist, the processor creates it.

If the lookup table is static, you can configure the processor to load the table only once, enabling the processor to cache and reuse the data for the duration of the pipeline run.

If not loading only once, and if the processor passes data to multiple stages, you might enable caching to improve pipeline performance.

You can optionally enable pushdown optimization and configure additional Snowflake properties.