Child pages
  • KIP-12 - Kafka Sasl/Kerberos and SSL implementation
Skip to end of metadata
Go to start of metadata


Current state: "Accepted"

Discussion thread: here

JIRA: here

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).


The goal is to add sasl authentication capability to Kafka brokers and provide ssl for encryption.

Public Interfaces

  • Channel wrapper for TransportLayer and AuthenticationLayer providing necessary handshake and authentication methods and also read(ByteBuffer buf) , write(ByteBuffer buf), write(ByteBuffer[] buf).
  • TransportLayer is an interface for network transportLayer.
  • PlainTextTransportLayer provides plain text socket channel methods
  • SSLTransportLayer provides ssl handshake and read/write methods.
  • Authenticator is an interface to providing client/server authentication.
  • SaslServerAuthenticationLayer implements AuthenticationLayer, provides authentication methods for server side.
  • SaslClientAuthenticationLayer implements AuthenticationLayer, provides client side authentication.

  • User: This class will be used to get the remoteUserId and add it to the Session Object (
  • KafkaPrincipalToLocalPlugin: This is a pluggable class with a default implementation which translates a kerberos principal which looks like "testuser/" to "testuser". Users can provide a their own customized version of PrincipalToLocalPlugin.
  • AuthUtils: This class will consists of any utilities needed for SASL and other auth related methods.
  • KerberosLoginFactory:  It will use jaas config to login and generates a subject. 
  • Protocol accepts the protocol type (PLAINTEXT, SSL , PLAINTEXT+SASL,  SSL+SASL)
    • PLAINTEXT (non-authenticated, non-encrypted)
      • This channel will provide exact behavior for communication channels as previous releases
    • SSL
      •  SSL  implementation. Authenticated principal in the session will be from the certificate presented or the peer host. 
      • SASL authentication will be used over plaintext channel. Once the sasl authentication established between client and server . Session will have client’s principal as authenticated user. There won’t be any wire encryption in this case as all the channel communication will be over plain text .
    • SASL+SSL
      • SSL will be established initially and  SASL authentication will be done over SSL. Once SASL authentication is established users principal will be used as authenticated user .  This option is useful if users want to use SASL authentication ( for example kerberos ) with wire encryption.



  • SecurityConfig , a config file for provider SecurityProtocol,  SSL config and SASL mechanisms.
  • BlockingChannel interface changes as it accepts the Protocol to create appropriate channels.

Proposed Changes

we will be using SASL to provide authentication and SSL to provider encryption in connection oriented protocols. 


As part of SASL implementation we will be using JAAS config to read kerberos ticket and authenticate. More info on JAAS Config

Proposed JAAS Login config file will look like this.

Users needs to pass as part of JVM params .

This JAAS file along with can be used to login into LDAP or KERBEROS etc.. 

Here are some details on LdapLoginModule for JAAS 

Jaas Config











Callbacks  for Login , SaslServer, SaslClient

    Callbacks for the above modules will help in grabbing the users authentication information.





SASL Authentication exchange


1) KafkaClient picks the principal it wants to use by looking at KafkaClient jaas config (example above).

2) Authenticates against KDC (kerberos) using the KafkaClient principal.

3) KafkaClient constructs the service principal name based on the jaas config serviceName

4) KafkaClient initiates challenge/response with KafkaBroker along with KafkaClient principal and service principal . Depending on the KafkaBroker response these challenge/response might continue until it receives COMPLETE from the KafkaBroker.


1) KafkaBroker will accept the connection and accepts the client and service principal

2) checks if the service principal is same as the one KafkaBroker running with.

3) KafkaBroker accepts/rejects the clients token

4) Returns the response to the client if its authenticated or not.

5) Once client is authenticated they can send Kafka requests.


Security Config

SecurityConfig will be shared across clients and brokers. If not provided communication channels fall back to PLAINTEXT . Here are proposed configs



Compatibility, Deprecation, and Migration Plan

As per previous security discussions and multiport work being done as part of this JIRA

JIRA Issues Macro: Unable to locate JIRA server for this macro. It may be due to Application Link configuration.

Users/Clients can still communicate with non-secure/non-sasl kafka brokers.

Open Questions

1) Users should be configuring . after setting authentication.enable to true in server and also in clients. Any objections on this approach?

2) We need to pass SecurityConfig  to ReplicaManager -> ReplicaFetcherManager -> ReplicaFetcherThread -> SimpleConsumer -> BlockingChannel.  This is a necessary as BlockingChannel can based on the 

Protocol.securityProtocol initiate the appropriate Channel described earlier.  Any better approach passing down this information to BlockingChannel to create appropriate TransportLayer and AuthenticationLayer.

3) Similarly Producer and Consumer does KerberosLoginManager.init() with "KafkaClient" section  if authentication.enable set to true and also passes Protocol to BlockingChannel.  Any alternatives or objections on this approach?

Rejected Alternatives

1) Using GSS-API as an authentication layer and security layer. This can only be used with Kerberos.

2) Using SASL to provide encryption of network data. After talking to other projects like HDFS (HBASE as well) they noticed 10x slow down when encryption is enabled on SASL. Hence the reason to separate transportLayer and authentication layer . SASL as authentication users can choose PLAINTEXT for no encryption and SSL for encryption and still be performant.

3) Using TLS for kerberos authentication. .  In this case clients expects server to be running with "host/hostname@realm" keytab. Here "host" cannot be changed.

HDFS noticed security issues as well with TLS for kerberos.


  • No labels