KIP-35 - Retrieving protocol version introduced a mechanism for dynamically determining the functionality of a Kafka broker and KIP-97: Improved Kafka Client RPC Compatibility Policy introduced a new compatibility policy and guarantees for the Java client. Previously, the general rule was that the Kafka broker a client interacted with was required to be a newer version than the client. Now, newer versions of the Java clients and other clients that support KIP-35 such as librdkafka are able to fall back to older request types or throw appropriate errors if functionality is not available.
This flexibility is a big win for users, who now have much more flexibility in adopting new client versions without requiring broker upgrades first. This is especially beneficial for client-side only bug fixes or improvements that don't require any additional support, both in user applications and in libraries/frameworks like Kafka Connect and Kafka Streams. However, it also complicates compatibility, especially while users may still be running versions of the clients from before this support existed.
The goal of this page is to provide an easy reference to understand what clients will work with a given version of the broker. This document only includes information for brokers since the introduction of KIP-35, which was added in 0.10.0.0. For earlier versions, you can either assume the previous policy applies or refer to the specific client documentation for details of how it supports multiple broker versions (usually via configuration).
To keep the table simple, we only break compatibility down based on Java & KIP-35-enabled clients. Exact compatibility for KIP-35 clients will depend on when support was added. Here is the version when support was added for some popular clients:
Note that while clients have different versioning schemes, all the data here is based on Kafka releases. If versions differ, you'd need to map these to specific releases for the individual clients.