Stage Event Generation

You can configure certain stages to generate events. Event generation differs from stage to stage, based on the way the stage processes data. For details about each the event generation for each stage, see "Event Generation" in the stage documentation.

The following table lists event-generating stages and when they can generate events:
Stage Generates events when the stage...
Amazon S3 origin
  • Completes processing all available objects and the configured batch wait time has elapsed.

For more information, see Event Generation for the Amazon S3 origin.

Azure Data Lake Storage Gen1 origin
  • Starts processing a file.
  • Completes processing a file.
  • Completes processing all available files and the configured batch wait time has elapsed.

For more information, see Event Generation for the Azure Data Lake Storage Gen1 origin.

Azure Data Lake Storage Gen2 origin
  • Starts processing a file.
  • Completes processing a file.
  • Completes processing all available files and the configured batch wait time has elapsed.

For more information, see Event Generation for the Azure Data Lake Storage Gen2 origin.

Directory origin
  • Starts processing a file.
  • Completes processing a file.
  • Completes processing all available files and the configured batch wait time has elapsed.

For more information, see Event Generation for the Directory origin.

File Tail origin
  • Starts processing a file.
  • Completes processing a file.

For more information, see Event Generation for the File Tail origin.

Google BigQuery origin
  • Successfully completes a query.

For more information, see Event Generation for the Google BigQuery origin.

Google Cloud Storage origin
  • Completes processing all available objects and the configured batch wait time has elapsed.

For more information, see Event Generation for the Google Cloud Storage origin.

Groovy Scripting origin
  • Runs a script that generates events.

For more information, see Event Generation for the Groovy Scripting Origin.

Hadoop FS Standalone origin
  • Starts processing a file.
  • Completes processing a file.
  • Completes processing all available files and the configured batch wait time has elapsed.

For more information, see Event Generation for the Hadoop FS Standalone origin.

JavaScript Scripting origin
  • Runs a script that generates events.

For more information, see Event Generation for the JavaScript Scripting Origin.

JDBC Multitable Consumer origin
  • Completes processing the data returned by the queries for all tables.

For more information, see Event Generation for the JDBC Multitable Consumer origin.

JDBC Query Consumer origin
  • Completes processing all data returned by a query.
  • Successfully completes a query.
  • Fails to complete a query.

For more information, see Event Generation for the JDBC Query Consumer origin.

Jython Scripting origin
  • Runs a script that generates events.

For more information, see Event Generation for the Jython Scripting Origin.

MapR FS Standalone origin
  • Starts processing a file.
  • Completes processing a file.
  • Completes processing all available files and the configured batch wait time has elapsed.

For more information, see Event Generation for the MapR FS Standalone origin.

MongoDB origin
  • Completes processing all data returned by a query.

For more information, see Event Generation for the MongoDB origin.

Oracle Bulkload origin
  • Completes processing data in a table.

For more information, see Event Generation for the Oracle Bulkload origin.
Oracle CDC Client origin
  • Reads DDL statements in the redo log.

For more information, see Event Generation for the Oracle CDC Client origin.

Salesforce origin
  • Completes processing all data returned by a query.

For more information, see Event Generation for the Salesforce origin.

SAP HANA Query Consumer origin
  • Completes processing all data returned by a query.
  • Successfully completes a query.
  • Fails to complete a query.

For more information, see Event Generation for the SAP HANA Query Consumer origin.

SFTP/FTP/FTPS Client origin
  • Starts processing a file.
  • Completes processing a file.
  • Completes processing all available files and the configured batch wait time has elapsed.

For more information, see Event Generation for the SFTP/FTP/FTPS Client origin.
SQL Server 2019 BDC Multitable Consumer origin
  • Completes processing the data returned by the queries for all tables.

For more information, see Event Generation for the SQL Server 2019 BDC Multitable Consumer origin.

SQL Server CDC Client origin
  • Completes processing the data in the associated CDC tables.

  • When you enable checking for schema changes, the origin generates an event each time it detects a schema change.

For more information, see Event Generation for the SQL Server CDC Client origin.

SQL Server Change Tracking origin
  • Completes processing the data in all specified change tracking tables.

For more information, see Event Generation for the SQL Server Change Tracking origin.

Teradata Consumer origin
  • Completes processing the data returned by the queries for all tables.

For more information, see Event Generation for the Teradata Consumer origin.

Windowing Aggregator processor
  • Performs aggregations, based on the configured window type and time window.

For more information, see Event Generation for the Windowing Aggregator processor.

Groovy Evaluator processor
  • Runs a script that generates events.

For more information, see Event Generation for the Groovy Evaluator processor.

JavaScript Evaluator processor
  • Runs a script that generates events.

For more information, see Event Generation for the JavaScript Evaluator processor.

Jython Evaluator processor
  • Runs a script that generates events.

For more information, see Event Generation for the Jython Evaluator processor.

TensorFlow Evaluator processor
  • Evaluates the entire batch at once.

For more information, see Event Generation for the TensorFlow Evaluator processor.
Windowing Aggregator processor
  • Performs aggregations.

For more information, see Event Generation for the Windowing Aggregator processor.
Amazon S3 destination
  • Completes writing to an object.
  • Completes streaming a whole file.

For more information, see Event Generation for the Amazon S3 destination.

Azure Data Lake Storage (Legacy) destination
  • Closes a file.
  • Completes streaming a whole file.

For more information, see Event Generation for the Azure Data Lake Storage (Legacy) destination.

Azure Data Lake Storage Gen1 destination
  • Closes a file.
  • Completes streaming a whole file.

For more information, see Event Generation for the Azure Data Lake Storage Gen1 destination.

Azure Data Lake Storage Gen2 destination
  • Closes a file.
  • Completes streaming a whole file.

For more information, see Event Generation for the Azure Data Lake Storage Gen2 destination.

Google Cloud Storage destination
  • Completes writing to an object.
  • Completes streaming a whole file.

For more information, see Event Generation for the Google Cloud Storage destination.

Hadoop FS destination
  • Closes a file.
  • Completes streaming a whole file.

For more information, see Event Generation for the Hadoop FS destination.

Hive Metastore destination
  • Updates the Hive metastore by creating a table, adding columns, or creating a partition.
  • Generates and writes a new Avro schema file.

For more information, see Event Generation for the Hive Metastore destination.

Local FS destination
  • Closes a file.
  • Completes streaming a whole file.

For more information, see Event Generation for the Local FS destination.

MapR FS destination
  • Closes a file.
  • Completes streaming a whole file.

For more information, see Event Generation for the MapR FS destination.

SFTP/FTP/FTPS Client destination
  • Closes a file.
  • Completes streaming a whole file.

For more information, see Event Generation for the SFTP/FTP/FTPS Client destination.

Snowflake File Uploader destination
  • Completes streaming a whole file.

For more information, see Event Generation for the Snowflake File Uploader destination.
ADLS Gen1 File Metadata executor
  • Changes file metadata, such as the file name, location, or permissions.
  • Creates an empty file.
  • Removes a file or directory.

For more information, see Event Generation for the ADLS Gen1 File Metadata executor.
ADLS Gen2 File Metadata executor
  • Changes file metadata, such as the file name, location, or permissions.
  • Creates an empty file.
  • Removes a file or directory.

For more information, see Event Generation for the ADLS Gen2 File Metadata executor.
Amazon S3 executor
  • Creates a new Amazon S3 object.
  • Copies an object another location.
  • Adds tags to an existing object.

For more information, see Event Generation for the Amazon S3 executor.

Databricks Job Launcher executor
  • Starts a Databricks job.

For more information, see Event Generation for the Databricks Job Launcher executor.

Databricks Query executor
  • Determines that the submitted query completed successfully.
  • Determines that the submitted query failed to complete.

For more information, see Event Generation for the Databricks Query executor.
Google Cloud Storage executor
  • Creates a new Google Cloud Storage object.
  • Copies an object to another location.
  • Moves an object to another location.
  • Adds metadata to an existing object.
For more information, see Event Generation for the Google Cloud Storage executor.
HDFS File Metadata executor
  • Changes file metadata, such as the file name, location, or permissions.
  • Creates an empty file.
  • Removes a file or directory.

For more information, see Event Generation for the HDFS File Metadata executor.

Hive Query executor
  • Determines that the submitted query completed successfully.
  • Determines that the submitted query failed to complete.

For more information, see Event Generation for the Hive Query executor.

JDBC Query executor
  • Determines that the submitted query completed successfully.
  • Determines that the submitted query failed to complete.

For more information, see Event Generation for the JDBC Query executor.

MapR FS File Metadata executor
  • Changes file metadata, such as the file name, location, or permissions.
  • Creates an empty file.
  • Removes a file or directory.

For more information, see Event Generation for the MapR FS File Metadata executor.

MapReduce executor
  • Starts a MapReduce job.

For more information, see Event Generation for the MapReduce executor.

Snowflake executor
  • Determines that the submitted query completed successfully.
  • Determines that the submitted query failed to complete.

For more information, see Event Generation for the Snowflake executor.

Spark executor
  • Starts a Spark application.

For more information, see Event Generation for the Spark executor.