HDFS File Metadata

The HDFS File Metadata executor changes file metadata, creates an empty file, or removes a file or directory in HDFS or a local file system each time it receives an event. You cannot perform multiple tasks in the same executor. To perform more than one task, use additional executors. For information about supported versions, see Supported Systems and VersionsSupported Systems and Versions in the Data Collector documentation.

Use the HDFS File Metadata executor as part of an event stream. For example, you might use the executor to move a file or change file permissions after it receives a file closure event from the Hadoop FS destination.

You can use the executor in any logical way, such as changing file metadata after receiving file closure events from the Hadoop FS or Local FS destinations.

When changing metadata, you configure an expression that represents the location and name of the file to process, and then specify the changes you want to perform. When creating an empty file, you specify the output location for the file, and can optionally specify the owner, permissions, and ACLs for the file. When removing a file or directory, you specify the location of the file or directory.

When necessary, you can enable Kerberos authentication and specify an HDFS user. You can also use HDFS configuration files and add other HDFS configuration properties as needed.

You can also configure the executor to generate events for another event stream. For more information about dataflow triggers and the event framework, see Dataflow Triggers Overview.

For a solution that describes how to use the HDFS File Metadata executor, see Managing Output Files.