Column Remover

The Column Remover processor removes columns and passes the remaining columns downstream.

When you configure the Column Remover processor, you specify whether to keep or exclude the selected columns. You define the columns to act on by listing or selecting individual columns, or by specifying the strings, prefixes, or suffixes to match with column names. For more advanced column name patterns, you can use regular expressions to define groups of columns.

Example

Say you want to remove a set of columns that end with _prelim. To do this, you can configure the processor as follows:
  • Action: Remove Listed Columns
  • Column Match Criteria: Column Name Suffix
  • Suffix: _prelim

If some of the columns that you want to remove end with _preliminary, you might add _preliminary to the Suffix property.

Or, you could alternatively configure the processor to remove all columns that include _prelim anywhere in the column name, as follows:
  • Action: Remove Listed Columns
  • Column Match Criteria: Column Name Contains Text
  • String: _prelim

Configuring a Column Remover

Configure a Column Remover processor to remove columns from your data.

  1. On the General tab, configure the following properties:
    General Property Description
    Name Stage name.
    Description Optional description.
    Cache Data Caches processed data.
  2. Configure the following properties on the Remove tab:
    Remove Property Description
    Action Method for removing columns:
    • Remove Listed Columns
    • Keep Listed Columns
    Column Match Criteria Criteria to determine the columns to act on:
    • Column Name - Acts on one or more specified columns.
    • Column Name Contains Text - Acts on columns with names that include one of the specified strings.
    • Column Name Prefix - Acts on columns with names that start with one of the specified prefixes.
    • Column Name Suffix - Acts on columns with names that end with one of the specified suffixes.
    • Column Name Matches Regex - Acts on columns with names that match the results of one of the specified regular expressions.
    Important: To prevent removing inappropriate columns, when using any option besides Column Name, ensure that the specified details match only the columns that you want to act on.
    Columns Names of the columns to use.

    Available when matching based on column name.
    Strings Strings that column names must include.

    The processor acts on columns that match at least one of the specified strings.

    Available when matching based on text.
    Prefixes Prefixes, or set of characters, that column names must start with.

    The processor acts upon columns that include one of the specified prefixes.

    Available when matching based on prefixes.
    Suffixes Suffixes, or set of characters, that column names must end with.

    The processor acts upon columns that include one of the specified suffixes.

    Available when matching based on suffixes.
    Regular Expressions Regular expressions that column names must match.

    The processor acts upon columns that match the results of at least one of the specified expressions.

    Available when matching based on regular expressions.