Column Transformer

The Column Transformer processor transforms columns by performing row-based calculations on column values and writing the results to new or existing columns.

To create a SQL expression, you define the name of the column to receive the results of the expression. And then, you use Snowflake SQL to define the expression to use.

You can use Snowflake user-defined functions in expressions.

For more information about configuring expressions, see Expressions in Pipeline and Stage Properties.

Output Columns

When you configure a SQL expression, the Column Transformer processor writes the results of the expression to an output column. You can use an existing column or create a new one.

When you use an existing column, the processor replaces the incoming value with the new value. When you use a new column, the processor adds it to the row and passes the results of the expression.

Configuring a Column Transformer Processor

Configure a Column Transformer processor to perform row-based calculations on column values.

  1. On the General tab, configure the following properties:
    General Property Description
    Name Stage name.
    Description Optional description.
    Cache Data Caches processed data.
  2. To configure an expression, click the SQL Expression tab and configure the following information:
    SQL Expression Property Description
    Output Column Name

    Name of the column to contain the results of the expression. Enter the name of a new or existing column as follows: ColumnName.

    If you use an existing column, the processor replaces existing values.

    You can use the asterisk wildcard to represent array indices and map elements.

    SQL Expression SQL expression to evaluate.

    Optionally, click Ctrl + Space Bar for help with creating the expression.

    For more information about expressions, see Expressions in Pipeline and Stage Properties.
  3. Using simple or bulk edit mode, click the Add icon to add additional expressions.