Versions Compared

Old Version 6

changes.mady.by.user Rajini Sivaram

Saved on

compared with

New Version 7

changes.mady.by.user Rajini Sivaram

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

With the current Kafka SASL implementation, broker closes the client connection if SASL authentication fails without providing feedback to the client to indicate that authentication failed. Clients see this as a connection failure and do not get any feedback for the reason why the connection was closeda disconnection during authentication which may be related to authentication failure, but could also be due to broker failure. Client logs a warning with a hint that the disconnection was likely to have  been due to an authentication failure, but does not handle this as a fatal security exception since the error may not be due to an authentication failure. Producers and consumers retry, attempting to create successful connections, treating authentication failures as transient failures. There are no log entries on the client-side to indicate that any of these connection failures were due to authentication failure.

Goals

  1. Improve diagnostics for SASL authentication failures.
  2. Reduce retries when authentication fails. Authentication failures should be treated as non-retriable exceptions rather than transient failures.
  3. Reduce blocking in clients when authentication fails. If connection to a broker fails authentication, metadata wait operations in producers and consumers should avoid unnecessary blocking and throw an exception indicating authentication failure.

...

  • ApiVersions request is sent by the client to determine SaslHandshake version prior to authentication. This is supported by 0.10/ 0.11 brokers, retaining compatibility between 1.0.0 clients and older brokers. Since 0.10/0.11 brokers treat schema exceptions in the first request as GSSAPI auth message for compatibility with 0.9.x clients, ApiVersions v0 request will be used. As no throttling is performed prior to authentication, v0 is sufficient for the initial request.
  • If SaslHandshake version is v0, then the older wire format is used for authentication. SASL authentication messages are sent as length-encoded byte arrays without wrapping the messages with Kafka protocol headers.
  • If SaslHandshake version is v1, clients will send SASL authentication messages using the new SaslAuthenticate request/response format, where the actual SASL authentication message blobs are wrapped in the Kafka protocol. The network layer code for SASL already does creation and parsing of Kafka protocol requests and responses to process SaslHandshake request, so this will be a straightforward change.
  • The error code in the final message from the broker will indicate if authentication succeeded or failed. In the case of authentication failure, a SaslAuthenticate response will be sent with error_code indicating authentication failure and empty sasl_auth_bytes before the connection is closed by the broker. Successful authentications will be logged at debug level. Failed authentications will be logged as warnings.

Authentication error handling

  • If authentication fails, broker will return an error response and then close the connection.
  • Java clients will log a warning for authentication failures, distinguishing these from EOF exceptions due to connection failure.
  • Nodes to which connection failed due to authentication failure will be blacked out for the reconnect backoff interval.
  • Producer waitForMetadata and consumer ensureCoordinatorReady will be updated to throw AuthenticationFailedException if connection to a broker fails authentication.

Versioning of SaslHandshake and SaslAuthenticate requests

  • SaslHandshake version will be bumped up from 0 to 1 without any changes to the schema. SaslHandshake v1 indicates that broker supports SaslAuthenticate requests.
  • For future versions, SaslHandshake and SaslAuthenticate requests versions may be updated independently. ApiVersions request will be sent prior to SASL authentication to determine versions supported for SaslHandshake and SaslAuthenticate.
  • SaslHandshake request version will be bumped up if there are any changes required to the protocol used to select SASL mechanisms.
  • SaslAuthenticate request version will be bumped up if any additional field is added to the requests or responses, e.g. to support new error paths. Version may also be bumped up if new error codes are added which require special handling.
  • SaslHandshake and SaslAuthenticate request versions will not be bumped up to enable new mechanisms unless the new mechanism requires schema/protocol changes to the existing SaslHandshake/SaslAuthenticate request/response format.

Compatibility, Deprecation, and Migration Plan

...