Status
Current state: Voting In Progress
Discussion thread: here
Vote thread: here
JIRA:
-
KAFKA-7089Getting issue details...
STATUS
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Motivation
The consumer group command, when used for describing a consumer group in the default or --offsets
mode, currently reports current offset, log end offset, and the lag for each topic partition the group has offset for. It would be useful to extend the tool to also report the beginning offset for each partition. They could help with determining where the group stands with respect to consuming from the partition, whether it is consuming OK or is falling behind, what percentage of messages are consumed, whether the partition is empty, etc. The beginning or earliest log offset is not something that can be derived from the provided info.
Public Interfaces
The consumer group tool will be updated for --describe
and --describe --offsets
options by adding a LOG-START-OFFSET
column.
Proposed Changes
No protocol change is required as the log start offset is readily available (similar to how log end offset is retrieved). The output of the command for --describe
and --describe --offsets
options will change from something like this:
$ bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group my-group TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID HOST CLIENT-ID my-topic 0 16577 33154 16577 consumer1-81fe8a2a-0b29-4964-a1d4-4741582132bb /127.0.0.1 consumer1 my-topic 1 16578 33155 16577 consumer1-81fe8a2a-0b29-4964-a1d4-4741582132bb /127.0.0.1 consumer1 my-topic 2 16577 33155 16578 consumer2-db6f12f2-e1de-4bb7-93ec-6170fefa8830 /127.0.0.1 consumer2 my-second-topic 0 33023 44543 11520 consumer7-c1d056fa-eabc-42ed-b557-c848f747ec49 /127.0.0.1 consumer7 my-other-topic 1 7901 8652 751 - - - my-other-topic 0 7902 8654 752 - - -
to something like this:
$ bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group my-group TOPIC PARTITION CURRENT-OFFSET LOG-START-OFFSET LOG-END-OFFSET LAG CONSUMER-ID HOST CLIENT-ID my-topic 0 16577 9856 33154 16577 consumer1-81fe8a2a-0b29-4964-a1d4-4741582132bb /127.0.0.1 consumer1 my-topic 1 16578 9828 33155 16577 consumer1-81fe8a2a-0b29-4964-a1d4-4741582132bb /127.0.0.1 consumer1 my-topic 2 16577 9847 33155 16578 consumer2-db6f12f2-e1de-4bb7-93ec-6170fefa8830 /127.0.0.1 consumer2 my-second-topic 0 33023 18484 44543 11520 consumer7-c1d056fa-eabc-42ed-b557-c848f747ec49 /127.0.0.1 consumer7 my-other-topic 1 7901 0 8652 751 - - - my-other-topic 0 7902 0 8654 752 - - -
Compatibility, Deprecation, and Migration Plan
Users who use the consumer group command (e.g. in custom tools) and somehow rely on parsing the output of the --describe
or --describe --offsets
options of the command may have to adjust their clients to account for this new column.
Rejected Alternatives
N/A.