Pipeline Finisher

When the Pipeline Finisher executor receives an event, the executor stops a pipeline and transitions it to a Finished state. This allows the pipeline to complete all expected processing before stopping.

Use the Pipeline Finisher executor as part of an event stream. You can use the Pipeline Finisher executor in any logical way, such as stopping a pipeline upon receiving a no-more-data event from the JDBC Query Consumer origin.

For example, you might use the executor in a pipeline designed to migrate all existing data from Microsoft SQL Server to HDFS. And then use a separate pipeline to process incremental updates. Or, you might use the executor to perform traditional "batch" processing - to process data, then stop when all data is processed rather than waiting indefinitely for more data.

When you configure a Pipeline Finisher executor, you can specify whether the executor should reset the origin after each pipeline run. When needed, you can use a precondition to limit the records that enter the stage to stop the pipeline. You might also configure the pipeline to notify you when the Pipeline Finisher executor stops the pipeline.

Before using the Pipeline Finisher executor, review the recommended implementation information.

For a solution that describes how to use the Pipeline Finisher executor, see Stopping a Pipeline After Processing All Available Data. For more information about dataflow triggers and the event framework, see Dataflow Triggers Overview.