Field Order
The Field Order processor orders fields in a map or list-map field and outputs the fields into a list-map or list root field.
Use the Field Order processor for the following reasons:
- Order the fields in a map field
- Fields in a map field have no defined order. Use the Field Order processor to specify the order of the fields and then output the fields to either the list-map or list root field which both retain field order.
- Reorder the fields in a list-map field
- Fields in a list-map field have a defined order. Use the Field Order processor when you need to reorder the fields, and then output the fields to the list-map root field.
When you configure the Field Order processor, you define the order of the fields. You specify whether to output the ordered fields to a list-map or list root field.
You configure how the processor handles missing or extra fields in the record. You can also list any fields that you want the processor to discard from the record.
Missing and Extra Fields
When you configure the Field Order processor, you specify the list of fields to order. An incoming record might be missing one of those fields or it might have some extra fields. You configure how the processor handles any missing or extra fields.
The Field Order processor can encounter the following types of fields in each record:
- Missing fields
- Missing fields are fields that you specified in the ordered list but that do not exist in the record. For example, you list these fields in the following order: FirstName, LastName, Age. However, an incoming record doesn't include the Age field.
- Extra fields
- Extra fields are fields that you did not specify in the ordered list but that do exist in the record. For example, you list these fields in the following order: FirstName, LastName, Age. However, an incoming record includes a MiddleName field.
Configuring a Field Order Processor
Configure a Field Order processor to order fields in a map or list-map field.
-
In the Properties panel, on the General tab, configure the
following properties:
General Property Description Name Stage name. Description Optional description. Required Fields Fields that must include data for the record to be passed into the stage. Tip: You might include fields that the stage uses.Records that do not include all required fields are processed based on the error handling configured for the pipeline.
Preconditions Conditions that must evaluate to TRUE to allow a record to enter the stage for processing. Click Add to create additional preconditions. Records that do not meet all preconditions are processed based on the error handling configured for the stage.
On Record Error Error record handling for the stage: - Discard - Discards the record.
- Send to Error - Sends the record to the pipeline for error handling.
- Stop Pipeline - Stops the pipeline. Not valid for cluster pipelines.
-
On the Order tab, configure the following
properties:
Order Property Description Fields to Order List of fields in the desired order. Type the fields in the list, or select the fields using preview data.
Output Type Root field type to output the ordered fields to: list-map or list. Missing Fields Action to take if a field in the ordered list is missing in a record: - Send record to error
- Use default value
If multiple fields are missing, the processor assigns the same default value and data type to each field.
Default Value Default value to use for missing fields. Default Type Default data type to use for missing fields. Extra Fields Action to take if the record includes an extra field that is not specified in the ordered list: - Send record to error
- Discard all extra fields
Discard Fields List of fields to discard from the record. Type the fields in the list, or select the fields using preview data.
These fields cannot also be specified in the ordered list. This field does not display if the processor is configured to discard all extra fields.