Event Generation

The Directory origin can generate events that you can use in an event stream. When you enable event generation, the origin generates event records each time the origin starts or completes reading a file. It can also generate events when it completes processing all available data and the configured batch wait time has elapsed.

Directory events can be used in any logical way. For example:

With the Pipeline Finisher executor to stop the pipeline and transition the pipeline to a Finished state when the origin completes processing available data.
When you restart a pipeline stopped by the Pipeline Finisher executor, the origin continues processing from the last-saved offset unless you reset the origin.
For an example, see Stopping a Pipeline After Processing All Available Data.
With the Email executor to send a custom email after receiving an event.
For an example, see Sending Email During Pipeline Processing.

With a destination to store event information.
For an example, see Preserving an Audit Trail of Events.

In Data Collector Edge pipelines, you can only pass events to destinations for storage. Since Data Collector Edge pipelines do not support executor stages, you cannot use events to trigger tasks.

For more information about dataflow triggers and the event framework, see Dataflow Triggers Overview.