Event Generation

The Hive Metastore destination can generate events that you can use in an event stream. When you enable event generation, the destination creates event records each time it updates the Hive metastore, including when it creates a table, adds columns, or creates a partition. It also creates events when it generates and writes a new Avro schema file to the destination system.

Note: Updates to existing tables, columns and partitions are treated as creates since the destination does not change or delete existing structures.

Hive Metastore events can be used in any logical way. For example:

With the Hive Query executor to run Hive or Impala queries after updating the Hive metastore.
For an example, see Automating Impala Metadata Updates for Drift Synchronization for Hive.
With the HDFS File Metadata executor to move or change permissions on closed files.
For an example, see Managing Output Files.
With the Email executor to send a custom email after receiving an event.
For an example, see Sending Email During Pipeline Processing.
With a destination to store event information.
For an example, see Preserving an Audit Trail of Events.

For more information about dataflow triggers and the event framework, see Dataflow Triggers Overview.