Master KIP
KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum (Accepted)
Status
Current state: Accepted
Discussion thread:
JIRA: KAFKA-9397
Motivation
As part of KIP-500, we would like to remove direct ZooKeeper access from the Kafka Administrative tools. We have many motivations for doing this. It improves security, decouples the server-side metadata format from the client, and is a necessary first step towards storing Kafka metadata in Kafka.
Before we can remove the --zookeeper flag from these tools, however, we need to first deprecate it. This KIP is about that deprecation process.
Public Interfaces
| Command Name | Status | Changes Needed |
|---|---|---|
| kafka-acls.sh | Does not support --zookeeper | none |
| kafka-broker-api-versions.sh | Does not support --zookeeper | none |
| kafka-configs.sh | Supports both --zookeeper and --bootstrap-server | deprecate --zookeeper |
| kafka-consumer-groups.sh | Does not support --zookeeper | none |
| kafka-delegation-tokens.sh | Does not support --zookeeper | none |
| kafka-delete-records.sh | Does not support --zookeeper | none |
| kafka-dump-log.sh | Does not support --zookeeper | none |
| kafka-leader-election.sh | Supports both --zookeeper and --bootstrap-server, but --zookeeper is already deprecated | none |
| kafka-log-dirs.sh | Does not support --zookeeper | none |
| kafka-preferred-leader-election.sh | Supports both --zookeeper and --bootstrap-server, but --zookeeper is already deprecated | none |
| kafka-reassign-partitions.sh | Supports only --zookeeper now, but --bootstrap-server support is planned as part of the accepted KIP-455 | deprecate --zookeeper |
| kafka-replica-verification.sh | Does not support --zookeeper | none |
| kafka-topics.sh | Supports both --zookeeper and --bootstrap-server, but --zookeeper is already deprecated | none |
| zookeeper-security-migration.sh | Supports only zookeeper | none |
| zookeeper-shell.sh | Supports only zookeeper | none |
Proposed Changes
For the kafka-configs.sh and kafka-reassign-partitions.sh commands, the --zookeeper option description should begin with "DEPRECATED".
If the --zookeeper option is used, the following should be printed on the console:
Warning: --zookeeper is deprecated and will be removed in a future version of Kafka. Use --bootstrap-server instead to specify a broker to connect to.
zookeeper-security-migration.sh and zookeeper-shell.sh won't be changed, since their functions are tightly tied to administering ZooKeeper itself.
Compatibility, Deprecation, and Migration Plan
Adding the new deprecations is a compatible operation.
Rejected Alternatives
There is no rejected alternatives.