Column Type Converter

The Column Type Converter processor converts columns to compatible data types.

When you configure the Column Type Converter processor, you can select or specify the column to convert, or you can define a regular expression that matches a set of columns to convert. Then, you specify the data type to convert to. You can configure multiple sets of conversions in a processor.

Example

Say you have several columns that end with _ID that you want to change from different numeric formats to an alphanumeric format. To do this, you can configure the processor as follows:
  • Column property: .+_ID
  • Target Type property: String

These settings convert columns such as User_ID, Item_ID, and Store_ID from their respective data types to String.

Column Conversion

When you configure the Column Type Converter processor, you can specify a column or regular expression to define the columns to convert.

The processor performs only valid type conversions. For more information about valid data type conversions, see the Snowflake documentation.

If you define multiple sets of column type conversion properties and if the same column is included in multiple sets, then the processor converts the column based on the first set of properties where it is included.

For example, say you configure the following sets of column type conversion properties, and the data includes a Store_ID column:
  • The first set converts all columns that end with _ID to the String data type:
    • Column property: .+_ID
    • Target Type property: String
  • The second set converts all columns that start with Store_ to the Integer data type:
    • Column property: Store_.+
    • Target Type property: Integer

With this order of properties, the processor converts the Store_ID column to String, because that is the first set of properties that the Store_ID column matches.

If you reverse the order of the sets of properties, then the processor converts the Store_ID column to Integer.

Configuring a Column Type Converter

Configure a Column Type Converter processor to convert columns to specified data types.

  1. On the General tab, configure the following properties:
    General Property Description
    Name Stage name.
    Description Optional description.
    Cache Data Caches processed data.
  2. On the Conversions tab, configure the following properties:
    Conversions Property Description
    Column Column to convert.

    Select or specify the column to convert. You can enter column names or select columns from preview data.

    Or, enter a regular expression that matches the names of the columns to convert.
    Target Type Data type to convert to.

    Select a valid data type for the conversion. For information about valid type conversions, see the Snowflake documentation.