Apache Kafka version 3.8.0 is shipped with an preview of the new generation of the consumer rebalance protocol (KIP-848). The next generation of the consumer rebalance protocol has three major changes: (1) the new consumer rebalance protocol; (2) a brand new group coordinator; and (3) a brand new threading model for the consumer. It is not recommended for production environments. Instead, we advice users to test it in clusters created for testing the feature.

Limitations:

  • The server side regular expression is not implemented yet. However, the regular client side regex subscription works.
  • The client side assignors are not supported yet.  

How to test it?

The simplest way is to create a new KRaft cluster following the instruction here but with the additional configurations added to config/kraft/server.properties:

group.coordinator.rebalance.protocols=classic,consumer

Then, it is possible to configure the kafka-console-consumer.sh command line tool to use it as follow:

bin/kafka-console-consumer.sh --topic <TOPIC> --from-beginning --bootstrap-server localhost:9092 --consumer-property group.protocol=consumer

It is also possible to configure the KafkaConsumer by setting the group.protocol property to consumer.

final Map<String, Object> configs = new HashMap<>();
configs.put(ConsumerConfig.GROUP_PROTOCOL_CONFIG, "consumer");
final ConsumerConfig consumerConfig = new ConsumerConfig(configs);

How to report issues?

Feel free to file a bug in the Apache Kafka project for any discovered issue. We appreciate your help to test it. Happy testing!

  • No labels