| Table of Contents |
|---|
Status
Current state: Under Discussion
...
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Motivation
- User gets offsets for one topic only. User needs to invoke GetOffsetShell for each topic separately. In order to get offsets of all topics, user first has to obtain the list of topics (by means of another tool) and then query each topic.
- GetOffsetShell does not have a convenient shell launcher as other console tools. It can be run only with a generic tool: kafka-run-class.sh.
- GetOffsetShell is inconsistent with other command-line tools in terms of command-line arguments. For example, --broker-list instead of --bootstrap-servers. (This is handled by KIP-499 and KAFKA-8507.)
- Users cannot provide additional consumer configurations for the tool, which makes it impossible to use with secure configurations.
- The tool still accepts deprecated and ignored arguments, which can be removed (offsets and max-wait-ms).
Public Interfaces
This KIP introduces a new command line tool: kafka-get-offsets.sh. The tool provides the following arguments:
...
- --topic - Can still be used as a simple topic name.
- --exclude-internal-topics - If omitted, the old behavior is achieved.
- Deprecated arguments were ignored and printing warnings for longer than the expected deprecation period.
- Other arguments did not change, or they are newly added and optional.
Proposed Changes
New implementation of GetOffsetShell does not change the interactions between the tool and the brokers, only adds additional filtering options in the tool.
...
Additionally, users can provide configuration overrides for the consumer, with the mechanics matching the ConsoleConsumer.
Compatibility, Deprecation, and Migration Plan
Any client tools depending on deprecated command-line arguments will stop working. This is acceptable, since the deprecated arguments are ignored for a long time now, and the tool was clearly indicating this for users.
Rejected Alternatives
For the public interface, a different flag was considered for controlling the exclusion of internal topics: --include-internal-topics. This would have also meant that the default behavior would exclude internal topics from querying. This would have broken backward-compatibility, as the old tool does not need any additional configuration to query internal topics.