Overview

You can preview data to help build or fine-tune a pipeline. You can preview complete or incomplete pipelines.

When you preview data, source data from the origin passes through the pipeline, allowing you to review how the data passes and changes through each stage. You can edit stage properties and run the preview again to see how your changes affect the data.

You can choose to preview using Spark libraries embedded in the Transformer installation or using the Spark cluster configured for the pipeline.

You can preview all stages in the pipeline, or you can perform a partial preview when one of the pipeline stages encounters an error. You can preview data for one stage at a time or for a group of stages. You can also view the preview data in list or table view.

When previewing data for a processor, you can choose how to display the order of output records. You can display output records in the order that matches the input records or in the order produced by the processor.

After running preview, you can view the input and output schema for each stage on the Schema tab in the pipeline properties panel.