Status
Current state: Approved
Discussion thread: here
Voting thread: here
JIRA: KAFKA-13351
PR: #11456
Motivation
Currently it is not possible to add headers to records when using the kafka-console-producer. The kafka-console-producer is an important debugging tool: The possibility to add headers to a record will improve the debugging experience.
Public Interfaces
New properties to LineMessageReader
Property | Valid Values | Default Value | Description |
---|---|---|---|
parse.headers | "true" or "false" | "false" | parse header information |
headers.delimiter | string | "\t" | delimits where header input for each line ends |
headers.separator | string | "," | separator between header key-value-pair |
headers.key.separator | string | ":" | separates the header's key from the header's value |
Default parsing pattern
when parse.headers=true and parse.key=true:
"h1:v1,h2...\tkey\tvalue"
when parse.headers=true and parse.key=false:
"h1:v1,h2...\tvalue"
when parse.headers=false and parse.key=true:
"key\tvalue"
when parse.headers=false and parse.key=false :
"value"
$ ./bin/kafka-console-producer.sh --bootstrap-server localhost:29092 --topic test --property parse.key=true --property parse.headers=true >h0:v0,h1:v1 key value
Proposed Changes
Add new properties to ConsoleProducer.LineMessageReader as per the above:
- parse.headers
- headers.delimiter
- headers.separator
- headers.key.separator
Compatibility, Deprecation, and Migration Plan
- What impact (if any) will there be on existing users?
The change is backward compatible. Current users of the kafka-console-producer can use it as they did so far.
- If we are changing behavior how will we phase out the older behavior?
Not applicable.
Rejected Alternatives
- Use other console producer like 'kafkacat'. This introduces additional dependencies whereas the proposed solution does not.