You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Next »

Status

Current state: Work in Progress

Discussion thread: here

JIRA: KAFKA-16143 - Getting issue details... STATUS

Motivation

The new consumer group rebalance protocol introduced in KIP-848 gave rise to a new consumer implementation to support the new protocol. The new protocol is referred to as CONSUMER, whereas the old protocol is called CLASSIC. Similarly, the new consumer implementation, AsyncKafkaConsumer, uses the CONSUMER protocol, while the older LegacyKafkaConsumer uses the CLASSIC protocol. If using the CLASSIC protocol, the proposed metrics below will be set to null or 0.

The new AsyncKafkaConsumer that uses the new CONSUMER rebalance protocol has some major design differences to the old consumer implementation. Namely, the threading model that is being used. The new consumer uses two threads; one thread for handling API calls and the other is a background thread that handles network IO with the broker. The addition of the proposed metrics would assist in debugging, providing a more insightful view of CONSUMER, and having a more holistic set of metrics for the consumer.

  • No labels