Kafka Message Keys

You can configure the Kafka Consumer and Kafka Multitopic Consumer origins to capture the message keys included in each Kafka message and store them in generated records. Kafka message keys can be string values or Avro messages, depending on how your Kafka system is configured.

Store message key values in a record when you want to use the values in pipeline processing logic or to write them to destination systems. When you store message key values, you can also configure a Kafka Producer destination to pass those values to Kafka as Kafka message keys. This allows Kafka to reuse the original message keys instead of generating new message keys for the data.

If you have no need for message keys, you can discard them. The Kafka Consumer and Kafka Multitopic Consumer origins discardKafka Multitopic Consumer origin discards message keys by default.