Passing Key Values to Kafka

You can configure the Kafka Producer destination to pass message key values stored in record header attributes to Kafka to be used as message keys. This enables Kafka to use existing message keys instead of creating new keys for the records.

To configure the destination to pass message key values to Kafka, configure the following properties:

Kafka Message Key property on the Kafka tab - Define an expression that indicates the location of the message key values in the records.
Message Key Format property on the Data format tab - Select the format of the message key values: String or Avro.

Tip: When you select the Message Key Format property, the destination provides a default expression in the Kafka Message Key property that you can edit as needed.

The expression that you use in the Kafka Message Key property depends on the data format of the original message keys:

String

For string message keys, use the following expression for the Kafka Message Key property:

${record:attribute('<message key attribute name>'}

Specify the name of the attribute where message key values are stored, as defined in the Kafka origin. By default, Kafka origins use the following attribute name: kafkaMessageKey.

Avro

For Avro message keys, use the following expression for the Kafka Message Key property:

${avro:decode(record:attribute('avroKeySchema'),base64:decodeBytes(record:attribute('<messsage key attribute name')))}

The first part of the expression defines the location of the schema for Avro message keys. Do not change this part of the expression.

The second part of the expression defines where the message key values are stored. Specify the name of the attribute where message key values are stored, as defined in the Kafka origin. By default, Kafka origins use the following attribute name: kafkaMessageKey.