Master KIP

KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum (Accepted)

Status

Current state: Accepted

Discussion thread:

JIRA: KAFKA-9397

Motivation

As part of KIP-500, we would like to remove direct ZooKeeper access from the Kafka Administrative tools.  We have many motivations for doing this.  It improves security, decouples the server-side metadata format from the client, and is a necessary first step towards storing Kafka metadata in Kafka.

Before we can remove the --zookeeper flag from these tools, however, we need to first deprecate it.  This KIP is about that deprecation process.

Public Interfaces

Command NameStatusChanges Needed
kafka-acls.shDoes not support --zookeepernone
kafka-broker-api-versions.shDoes not support --zookeepernone
kafka-configs.shSupports both --zookeeper and --bootstrap-serverdeprecate --zookeeper
kafka-consumer-groups.shDoes not support --zookeepernone
kafka-delegation-tokens.shDoes not support --zookeepernone
kafka-delete-records.shDoes not support --zookeepernone
kafka-dump-log.shDoes not support --zookeepernone
kafka-leader-election.shSupports both --zookeeper and --bootstrap-server, but --zookeeper is already deprecatednone
kafka-log-dirs.shDoes not support --zookeepernone
kafka-preferred-leader-election.shSupports both --zookeeper and --bootstrap-server, but --zookeeper is already deprecatednone
kafka-reassign-partitions.shSupports only --zookeeper now, but --bootstrap-server support is planned as part of the accepted KIP-455deprecate --zookeeper
kafka-replica-verification.shDoes not support --zookeepernone
kafka-topics.shSupports both --zookeeper and --bootstrap-server, but --zookeeper is already deprecatednone
zookeeper-security-migration.shSupports only zookeepernone
zookeeper-shell.shSupports only zookeepernone

Proposed Changes

For the kafka-configs.sh and kafka-reassign-partitions.sh commands, the --zookeeper option description should begin with "DEPRECATED".

If the --zookeeper option is used, the following should be printed on the console:

Warning: --zookeeper is deprecated and will be removed in a future version of Kafka.
Use --bootstrap-server instead to specify a broker to connect to.

zookeeper-security-migration.sh and zookeeper-shell.sh won't be changed, since their functions are tightly tied to administering ZooKeeper itself.

Compatibility, Deprecation, and Migration Plan

Adding the new deprecations is a compatible operation.

Rejected Alternatives

There is no rejected alternatives.


  • No labels