Apache Kafka version 3.78.0 is shipped with an early access preview of the new generation of the consumer rebalance protocol (KIP-848). The next generation of the consumer rebalance protocol has three major changes: (1) the new consumer rebalance protocol; (2) a brand new group coordinator; and (3) a brand new threading model for the consumer. It is not recommended for production environments. Instead, we advice users to test it in clusters created for testing the feature.
Limitations:
- The server side regular expression is not implemented yet. However, the regular client side regex subscription works.
- The client side assignors are not supported yet.
- We don't plan to support upgrading clusters using the early access feature. We are still actively developing the feature and we want to keep the flexibility for doing non-backward compatible changes.
- The
kafka-consumer-groups.shcommand line tool is not able to describe consumer groups using the new protocol. - Subscribing with regular expressions is not supported.
- The transaction verification (KIP-890) is not supported and must be disabled.
How to test it?
The simplest way is to create a new KRaft cluster following the instruction here but with the additional configurations added to config/kraft/server.properties:
| Code Block |
|---|
group.coordinator.rebalance.protocols=classic,consumer
transaction.partition.verification.enable=false |
Then, it is possible to configure the kafka-console-consumer.sh command line tool to use it as follow:
...