Status
Current state: "Accepted"
Discussion thread: here
JIRA: KAFKA-4259
Released: 0.10.2.0
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Motivation
Kafka currently uses JAAS login configuration files for SASL authentication. The login configuration can be supplied in one of the following ways:
- System property
java.security.auth.login.config
- Java security properties file (
lib/security/java.security
of the JRE), propertylogin.config.url.n
- Programmatically by invoking
javax.security.auth.login.Configuration.setConfiguration(Configuration)
1) is the most commonly used method (and the documented way) of configuring login context in Kafka. This requires access to a file containing the login configuration. Many applications which obtain credentials during runtime write a temporary file to use this approach (eg. https://developer.ibm.com/bluemix/2016/02/19/managing-iot-devices-with-kafka-and-mqtt/)
2) is typically not used by client applications since it requires write access to the JRE installation and modifies the configuration for all applications using the JRE.
3) is the only programmatic interface to set Configuration. This is a JVM-wide setting and is typically the cause of many bugs, including in Kafka tests. Programmatic creation of Configuration
objects is not very straightforward when credentials are known only at runtime. Also, the caching of JVM-wide Configuration
instance in Java makes it difficult to identify the source of bugs when configuration is updated dynamically.
To enable cloud applications to use Kafka without relying on a physical file to store credentials (eg. CloudFoundry applications without persistent storage), it will be useful to add a configuration option that simplifies JAAS configuration for Kafka clients. This will also be useful to configure multiple KafkaClient
login contexts when multiple users are supported within a JVM (KIP-83).
Public Interfaces
Configuration property
Name: sasl.jaas.config
Type: PASSWORD
(to prevent credentials from being logged eg. SASL/PLAIN password)
Format: The same format as a login context entry in JAAS configuration:
<LoginModuleClass> <ControlFlag> *(<OptionName>=<OptionValue>);
ControlFlag = required / requisite / sufficient / optional
Example:
org.apache.kafka.common.security.plain.PlainLoginModule required username=”alice” password=”alice-secret”;
This property will be enabled only in Kafka clients. Even though brokers can also use the same approach for setting JAAS configuration dynamically, there is currently no requirement to support dynamic JAAS configuration for brokers. This avoids the need to implement multiple login module support in sasl.jaas.config
. Broker-side implementation can easily be added in future if required.
sasl.jaas.config
will be of type PASSWORD
to avoid exposing credentials in logs. As with the current JAAS configuration files, this has the drawback that configuration issues cannot be debugged using log files alone. But this format was chosen to enable extensible mechanism-independent configuration that is consistent with the existing JAAS configuration.
Post Implementation Note: This config was added to brokers in 1.1.0 under KIP-226 - Dynamic Broker Configuration
Proposed Changes
Default JAAS configuration
If sasl.jaas.config
is not specified, the JAAS configuration instance configured in the JVM using any of the three methods described in the Motivation section is used. The implementation will return Configuration.getConfiguration()
that caches the configuration instance that is loaded. This is the current behavior.
JAAS configuration using client properties
If sasl.jaas.config
is specified, a new Configuration
instance is created with a single entry corresponding to the login type (KafkaClient
for clients). The generated Configuration
instance will not be set as the JVM-wide static value. Login context will be created using the constructor that accepts a configuration object rather than use the static value Configuration.getConfiguration()
.
LoginContext(String name, Subject subject, CallbackHandler callbackHandler, Configuration config)
When LoginManager
caching in Kafka is updated to support multiple users in a JVM (KIP-83 is addressing multiple users), sasl.jaas.config
can be set to different values for different clients to enable multiple users without manipulating JVM-wide Configuration
instances or adding additional mechanism-specific properties to identify users. All SASL mechanisms will be handled consistently using this approach.
Compatibility, Deprecation, and Migration Plan
- What impact (if any) will there be on existing users?
None
- If we are changing behavior how will we phase out the older behavior?
Existing behaviour will be retained as default.
Test Plan
Since most of the code path will be tested by existing tests, no additional system tests are required. Unit and integration tests will be added to ensure that JAAS configuration can be set and modified programmatically without changing JVM-wide values.
Rejected Alternatives
Add a configurable public interface for JaasConfigProvider
Rather than a property that enables the creation of a javax.security.auth.login.Configuration
instance, it would be sufficient to expose a configuration provider, leaving it up to applications how the configuration instance is created. This simplifies the Kafka implementation, but makes it harder for applications to generate configurations. Additional properties will be required to configure the provider.
Add separate properties for login module options instead of a composite property using jaas.conf format
The KIP proposes to use a single property sasl.jaas.config
containing all the options of a login context. Separate properties (eg. sasl.jaas,login.context, sasl.jaas.username, sasl.jaas.password etc.) may make it easier to parse the configuration. But since login option names are dependent on the login module class and the SASL mechanism, it is more flexible to use a single Kafka property in the standard JAAS format that captures the contents of a login context.