Union

The Union processor merges data from two or more input streams. All data must have the same schema.

The Union processor can generate different output based on the operation that you select:

Union - Passes all records from all incoming streams.
Intersect - Passes only the records that exist in all incoming streams.
Except - Passes only the records from Input 1 of the processor that do not have matching records from the other input streams.

When you configure the Union processor, you connect the upstream stages to the processor and then specify the operation to use.

When you use the Except operation, the input order of upstream stages is important. Stages are assigned to input streams based on the order that you connect them to the processor. To assign a stage to Input 1, connect it to the processor before any other stages. When working with two input streams, you can swap the inputs by clicking the processor in the canvas, and then clicking . Swapping is not available when the processor has more than two input streams.

Note: Due to metadata added to Transformer records within the pipeline, the Union processor does not display output when you preview pipelines. When including the processor in pipeline development, you must run the pipeline to review how the Union processor and subsequent stages process the data.