Field Remover

Supported pipeline types:

  • Data Collector

  • Data Collector Edge
The Field Remover processor removes fields from records. Use the processor to discard field data that you do not need in the pipeline.
When you configure the processor, you specify a list of fields and the action for the Field Remover processor to take with those fields:
  • Keep the listed fields and remove all other fields
  • Remove the listed fields
  • Remove any listed field that has a particular value:
    • Null
    • Empty string
    • Specified constant

Configuring a Field Remover Processor

Configure a Field Remover processor to remove unwanted fields from the pipeline.
  1. In the Properties panel, on the General tab, configure the following properties:
    General Property Description
    Name Stage name.
    Description Optional description.
    Required Fields Fields that must include data for the record to be passed into the stage.
    Tip: You might include fields that the stage uses.

    Records that do not include all required fields are processed based on the error handling configured for the pipeline.
    Preconditions Conditions that must evaluate to TRUE to allow a record to enter the stage for processing. Click Add to create additional preconditions.

    Records that do not meet all preconditions are processed based on the error handling configured for the stage.
    On Record Error Error record handling for the stage:
    • Discard - Discards the record.
    • Send to Error - Sends the record to the pipeline for error handling.
    • Stop Pipeline - Stops the pipeline. Not valid for cluster pipelines.
  2. On the Remove tab, configure the following properties:
    Field Remover Property Description
    Action Action to complete:
    • Keep listed fields
    • Remove listed fields
    • Remove any listed field if its value is null
    • Remove any listed field if its value is an empty string
    • Remove any listed field if its value is null or an empty string
    • Remove any listed field if its value is a specified constant
    Fields List of fields to keep or remove.

    You can use the asterisk wildcard to represent array indices and map elements.

    You can specify individual fields or use a field path expression to specify a set of fields.
    Constant Field value that results in removal of the field.

    Available when removing fields with specified constants.