Current state: Accepted
Discussion thread: here
JIRA: KAFKA-4060
Released: 0.10.2.0
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
This KIP removes the Zookeeper dependency from Kafka Streams. Currently, for Kafka Streams API accesses Zookeeper directly to create internal topics. We will use the client admin API introduced in KIP-4 to manage the internal topics via Kafka broker instead of directly accessing Zookeeper.
Zookeeper related config value, ZOOKEEPER_CONNECT_CONFIG in StreamsConfig, is deprecated.
We add a new KafkaStreams client which is used for internal topic management in Kafka Streams API. We will remove the dependency to Zookeeper from Kafka Streams API..
This feature is backward compatible from API perspective and no changes need to be done to streaming apps. However, if kafka cluster is secured the cluster admin should grant required privileges to the streaming app as described below.
Zookeeper related config value, ZOOKEEPER_CONNECT_CONFIG in StreamsConfig, is deprecated.
In the current version of Kafka Streams user app needs to have proper access privileges to the zookeeper in order to manage internal topics. After the changes proposed in this KIP user app will not need such privileges, however, user app will need new privileges in order to use Client Admin to manage the internal topics. Streaming application should have the following privileges for Client Admin:
User information need to be set the same way it is set in the streaming app if the cluster is secured.
The same tests that are currently used will be used for testing the new internal topic creation.
N/A.