Current state: "Accepted"
Discussion thread: here
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Kafka currently uses JAAS login configuration files for SASL authentication. The login configuration can be supplied in one of the following ways:
lib/security/java.securityof the JRE), property
1) is the most commonly used method (and the documented way) of configuring login context in Kafka. This requires access to a file containing the login configuration. Many applications which obtain credentials during runtime write a temporary file to use this approach (eg. https://developer.ibm.com/bluemix/2016/02/19/managing-iot-devices-with-kafka-and-mqtt/)
2) is typically not used by client applications since it requires write access to the JRE installation and modifies the configuration for all applications using the JRE.
3) is the only programmatic interface to set Configuration. This is a JVM-wide setting and is typically the cause of many bugs, including in Kafka tests. Programmatic creation of
Configuration objects is not very straightforward when credentials are known only at runtime. Also, the caching of JVM-wide
Configuration instance in Java makes it difficult to identify the source of bugs when configuration is updated dynamically.
To enable cloud applications to use Kafka without relying on a physical file to store credentials (eg. CloudFoundry applications without persistent storage), it will be useful to add a configuration option that simplifies JAAS configuration for Kafka clients. This will also be useful to configure multiple
KafkaClient login contexts when multiple users are supported within a JVM (KIP-83).
PASSWORD (to prevent credentials from being logged eg. SASL/PLAIN password)
Format: The same format as a login context entry in JAAS configuration:
<LoginModuleClass> <ControlFlag> *(<OptionName>=<OptionValue>);
ControlFlag = required / requisite / sufficient / optional
org.apache.kafka.common.security.plain.PlainLoginModule required username=”alice” password=”alice-secret”;
This property will be enabled only in Kafka clients. Even though brokers can also use the same approach for setting JAAS configuration dynamically, there is currently no requirement to support dynamic JAAS configuration for brokers. This avoids the need to implement multiple login module support in
sasl.jaas.config. Broker-side implementation can easily be added in future if required.
sasl.jaas.config will be of type
PASSWORD to avoid exposing credentials in logs. As with the current JAAS configuration files, this has the drawback that configuration issues cannot be debugged using log files alone. But this format was chosen to enable extensible mechanism-independent configuration that is consistent with the existing JAAS configuration.
Post Implementation Note: This config was added to brokers in 1.1.0 under KIP-226 - Dynamic Broker Configuration
sasl.jaas.config is not specified, the JAAS configuration instance configured in the JVM using any of the three methods described in the Motivation section is used. The implementation will return
Configuration.getConfiguration() that caches the configuration instance that is loaded. This is the current behavior.
sasl.jaas.config is specified, a new
Configuration instance is created with a single entry corresponding to the login type (
KafkaClient for clients). The generated
Configuration instance will not be set as the JVM-wide static value. Login context will be created using the constructor that accepts a configuration object rather than use the static value
LoginContext(String name, Subject subject, CallbackHandler callbackHandler, Configuration config)
LoginManager caching in Kafka is updated to support multiple users in a JVM (KIP-83 is addressing multiple users),
sasl.jaas.config can be set to different values for different clients to enable multiple users without manipulating JVM-wide
Configuration instances or adding additional mechanism-specific properties to identify users. All SASL mechanisms will be handled consistently using this approach.
Existing behaviour will be retained as default.
Since most of the code path will be tested by existing tests, no additional system tests are required. Unit and integration tests will be added to ensure that JAAS configuration can be set and modified programmatically without changing JVM-wide values.
Rather than a property that enables the creation of a
javax.security.auth.login.Configuration instance, it would be sufficient to expose a configuration provider, leaving it up to applications how the configuration instance is created. This simplifies the Kafka implementation, but makes it harder for applications to generate configurations. Additional properties will be required to configure the provider.
The KIP proposes to use a single property
sasl.jaas.config containing all the options of a login context. Separate properties (eg. sasl.jaas,login.context, sasl.jaas.username, sasl.jaas.password etc.) may make it easier to parse the configuration. But since login option names are dependent on the login module class and the SASL mechanism, it is more flexible to use a single Kafka property in the standard JAAS format that captures the contents of a login context.