Local FS

Supported pipeline types:
  • Data Collector

Use the Local FS destination to write records to files in a local file system.

When you configure a Local FS destination, you can define a directory template and time basis to determine the output directories that the destination creates and the files where records are written.

You can define a file prefix and suffix, the data time zone, and properties that define when the destination closes a file. You can specify the amount of time that a record can be written to its associated directory and what happens to late records.

You can write records to the specified directory, use the defined Avro schema, and roll files based on record header attributes. For more information, see Record Header Attributes for Record-Based Writes.

The destination can generate events for an event stream. For more information about the event framework, see Dataflow Triggers Overview.

You can use Gzip, Bzip2, Snappy, LZ4, and other compression formats to write output files.