Amazon S3

Supported pipeline types:

  • Data Collector

  • Data Collector Edge
The Amazon S3 destination writes data to Amazon S3. For information about supported versions, see Supported Systems and VersionsSupported Systems and Versions in the Data Collector documentation.

To write data to an Amazon Kinesis Firehose delivery system, use the Kinesis Firehose destination. To write data to Amazon Kinesis Streams, use the Kinesis Producer destination.

With the Amazon S3 destination, you configure the region, bucket, and common prefix to define where to write objects. You can use a partition prefix to specify the S3 partition to write to. You can also configure a prefix and suffix for the object name, and a time basis and data time zone for the stage.

You configure the authentication method that the destination uses to connect to Amazon S3.

The Amazon S3 destination can write data asynchronously to improve performance when writing to multiple prefixes. You can configure advanced properties to tune performance.

You can configure the destination to use Amazon Web Services server-side encryption to protect the data written to Amazon S3. You can also use a proxy user and compress data with gzip when writing to Amazon S3.

The Amazon S3 destination creates an object for each batch of data written to Amazon S3.

You can also use a connectionconnection to configure the destination.

The destination can generate events for an event stream. For more information about the event framework, see Dataflow Triggers Overview.