Event Generation
The event framework generates pipeline events for Data Collector standalone pipelines when the pipeline starts and stops.
In a Data Collector standalone pipeline, you can pass pipeline events to an executor or to another pipeline for additional processing. By default, these events are discarded. For more information about pipeline events, see Pipeline Event Generation.
For general information about the event framework, see Dataflow Triggers Overview.
Pipeline Event Records
Pipeline event records have the following event-related record header attributes. Record header attributes are stored as String values:
| Record Header Attribute | Description |
|---|---|
| sdc.event.type | Event type. Uses one of the following types:
|
| sdc.event.version | Integer that indicates the version of the event record type. |
| sdc.event.creation_timestamp | Epoch timestamp when the stage created the event. |
The event framework generates the following types of pipeline events:
- pipeline-start
- The event framework generates start events as the pipeline initializes, immediately after it starts and before individual stages are initialized.
- pipeline-stop
- The event framework generates stop events as the pipeline stops, either manually, programmatically, or due to a failure. The stop event is generated after all stages have completed processing and cleaning up temporary resources, such as removing temporary files.