Amazon S3

The Amazon S3 destination writes objects to Amazon S3. The Amazon S3 destination writes data based on the specified data format and creates a separate object for every partition.

The Amazon S3 destination writes to Amazon S3 using connection information stored in a Hadoop configuration file. Complete the prerequisite tasks before using the destination in a local pipeline.

When you configure the Amazon S3 destination, you specify the authentication method to use. You can also specify Amazon S3 server-side encryption for the data.

You can also use a connectionconnection to configure the destination.

You define the output location and write mode to use. When overwriting related partitions, first complete the overwrite partition requirement.

You select the data format to write and configure related properties. You can specify fields to use for partitioning files. You can also drop unrelated master records when using the destination as part of a slowly changing dimension pipeline.

You can also configure advanced properties such as performance-related properties and proxy server properties.