Object Metadata in Record Header Attributes

You can include Amazon S3 object metadata in record header attributes. Include metadata when you want to use the information to help process records. For example, you might include metadata if you want to route records to different branches of a pipeline based on the last-modified timestamp.

Use the Include Metadata property to include metadata in the record header attributes. When you include metadata in record header attributes, the Amazon S3 origin includes the following information:
System-defined metadata
The origin includes the following system-defined metadata:
  • Name - The object name. Bucket and prefix information is included as follows: 
    <bucket>/<prefix>/<object_name>
  • Cache-Control
  • Content-Disposition
  • Content-Encoding
  • Content-Length
  • Content-MD5
  • Content-Range
  • Content-Type
  • ETag
  • Expires
  • Last-Modified
For more information about Amazon S3 system-defined metadata, see the Amazon S3 documentation.
User-defined metadata
When available, the Amazon S3 origin also includes user-defined metadata in record header attributes.
Amazon S3 requires user-defined metadata to be named with the following prefix: x-amz-meta-.
When generating the record header attribute, the origin omits the prefix.
For example, if you have user-defined metadata called "x-amz-meta-extraInfo", the origin names the record header attribute as follows: extraInfo.

For more information about record header attributes, see Record Header Attributes.