Data Formats Overview

Data formats - such as Avro, JSON, and log - are methods to encode data that adhere to generally accepted specifications.

The way that stages process data can be similar based on the stage type and the type of data being processed. For example, file-based origins such as Directory and SFTP/FTP/FTPS Client will typically process data formats the same way. Similarly, message-based destinations such as Kafka Producer and JMS Producer generally process data formats the same way.

This chapter discusses how stages process data formats generally. For the details of how a stage processes different data formats, see the "Data Formats" section of the stage documentation.

For information about the data formats supported by each origin, processor, or destination, see Data Format Support.