Storing Message Keys

You can configure the Kafka Consumer and Kafka Multitopic Consumer originsKafka Multitopic Consumer origin to capture Kafka message keys and store them in records. Store message key values when you want to use them in the pipeline, write them to destinations, or pass them to Kafka to use as message keys. By default, the origins discard Kafka message keys.

Kafka origins can store message key values in the following locations:

In a specified record field: Store message key values in a specified field when you want to perform processing based on field values. When you store message key values in a field, you can access the values easily with expression-based and field-based stages, such as the Stream Selector or Field Replacer processors.; You might also store message key values in a field when you want to write message key values to destinations.
In a specified record header attribute: Store message key values in a specified record header attribute when you want to use the Kafka Producer to pass message key values to Kafka as Kafka message keys.; You can use record:attribute functions to access string message key values from within the pipeline. You cannot use functions to access Avro message key values in record header attributes because the Avro data is binary.; By default, record header attributes are not included in records when records are written to destination systems. To easily write message key values to destination systems, store the values in record fields.
In a specified record field and record header attribute: Store message key values in both a specified record field and a specified record header attribute when you have multiple uses for message key values.; This option enables easy access to message key values in record fields and allows passing message key values to Kafka as Kafka message keys.