Note that the current version of ZooKeeper (3.5.6 as of this writing and the version shipped with Apache Kafka 2.4) only supports mutual certificate authentication. There is a sever-side config "
ssl.clientAuth" that the ZooKeeper code recognizes (case-insensitively:
none are the valid options), but this config has no effect in 3.5.6 (
). A recent build from source confirms that this config worked in the 3.6 SNAPSHOT, but that version is not yet released This is fixed in version 3.5.7, which is the ZooKeeper version that will ship with Apache Kafka version 2.5.
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key ZOOKEEPER-3674
Note also that ZooKeeper will associate multiple identities with any session that successfully authenticates multiple ways (e.g. both client certificate and SASL). The X.509 identity is the full Distinguished Name from the client's certificate, and this can be changed (i.e. use just a part of the DN) only by implementing and using a custom ZooKeeper authentication provider that overrides the method
protected String getClientId(X509Certificate clientCert). A client that accesses an ACL-protected Znode is authorized if it has at least 1 of the identities present in any authorizing ACL.
- Harden/secure the configuration mechanism for Zookeeper TLS connectivity from:
- Kafka Brokers (including from
- Kafka Brokers (including from
- Support client certificate authentication to ZooKeeper both with and without SASL authentication in ZK Security Migrator and the broker (when
- Add system tests to confirm the hardened/secured configuration for TLS connectivity to ZooKeeper
- Add explicit Kafka documentation on how to configure TLS connectivity to ZooKeeperZooKeeper – both mutual TLS and encryption-only
- Add a reference in the Kafka documentation to the ZooKeeper Quorum TLS configuration (https://zookeeper.apache.org/doc/r3.5.67/zookeeperAdmin.html#Communication+using+the+Netty+framework)
Out of Scope
- Zookeeper-to-Zookeeper Quorum TLS system tests and in-depth documentation (the ZooKeeper project already has such tests and documentation)
- Dynamic reconfiguration of ZooKeeper TLS configs
- System tests to confirm the hardened/secured configuration for TLS connectivity to ZooKeeper
- The use of ZooKeeper Security Migrator and Kafka Brokers with client certificate authentication both with and without SASL
- Explicit Kafka documentation on how to configure TLS connectivity to ZooKeeperZooKeeper – both mutual TLS and encryption-only
Compatibility, Deprecation, and Migration Plan
- Migrating Zookeeper/Kafka clusters from non-TLS-enabled ZooKeeper to TLS-enabled ZooKeeper
- Invoking the Zookeeper Security Migration tool against TLS-enabled ZooKeeper both with and without ZK SASL authentication enabled
- TLS encryption-only (i.e. no client certificate) connectivity to ZooKeeper.
Compatibility testing is unnecessary because Zookeeper TLS is not available in prior versions.
The "zookeeper.ssl.context.supplier.class" configuration doesn't actually exist in ZooKeeper 3.5.67. The ZooKeeper admin guide documents it as being there, but it doesn't appear in the code. This means we can't support it in this KIP. It has been added in the latest ZooKeeper 3.6 SNAPSHOT, so this config could probably be added to Kafka via a new, small KIP if/when we upgrade to ZooKeeper 3.6 (which looks to be in release-candidate stage as of this writing).