...
The new consumer group rebalance protocol introduced in KIP-848 gave rise to a new consumer implementation to support the new protocol. The new protocol is referred to as CONSUMER, whereas the old protocol is called CLASSIC. Similarly, the new consumer implementation, AsyncKafkaConsumer, uses the CONSUMER protocol, while the older LegacyKafkaConsumer uses the CLASSIC protocol. If using the CLASSIC protocol, the proposed metrics below will be set to 0. This is because these metrics are not applicable when using to the CLASSIC protocolLegacyKafkaConsumer. In other words, the proposed metrics are only for the new consumer using the CONSUMER rebalance protocol.
...
We are proposing to add the below metrics to the consumer. These metrics will cover the new consumer components introduced in AsyncKafkaConsumer. This will give a more insightful set of metrics for those using the CONSUMER protocol.
Public Interfaces
New Metrics
Metric Name | Telemetry Metric Name | Data Type | Description | MBean |
|---|---|---|---|---|
time-between-network-thread-poll-max | org.apache.kafka.consumer.time.between.network.thread.poll.max | long | The maximum time taken, in milliseconds, between each poll in the network thread each time the thread is run | kafka.consumer:type=consumer-metrics,client-id={clientId} |
time-between-network-thread-poll-avg | org.apache.kafka.consumer.time.between.network.thread.poll.avg | long | The average time taken, in milliseconds, between each poll in the network thread each time the thread is run | kafka.consumer:type=consumer-metrics,client-id={clientId} |
application-event-queue-size | org.apache.kafka.consumer.application.event.queue.size | int | The current number of events in the consumer network application event queue | kafka.consumer:type=consumer-metrics,client-id={clientId} |
application-event-queue-age-avg | org.apache.kafka.consumer.application.event.queue.age.avg | long | The average time, in milliseconds, that application events are taking to be dequeued | kafka.consumer:type=consumer-metrics,client-id={clientId} |
application-event-processing-time | org.apache.kafka.consumer.application.event.processing.time | long | The average time, in milliseconds, that the consumer network takes to process all available events | kafka.consumer:type=consumer-metrics,client-id={clientId} |
unsent-requests-queue-size | org.apache.kafka.consumer.unsent.requests.size | int | The current number of unsent requests in the consumer network | kafka.consumer:type=consumer-metrics,client-id={clientId} |
unsent-requests-age-max | org.apache.kafka.consumer.unsent.requests.age.max | long | The maximum time, in milliseconds, that a request remained unsent in the consumer network | kafka.consumer:type=consumer-metrics,client-id={clientId} |
unsent-requests-age-avg | org.apache.kafka.consumer.unsent.requests.age.avg | long | The average time, in milliseconds, that requests are taking to be sent in the consumer network | kafka.consumer:type=consumer-metrics,client-id={clientId} |
background-event-queue-size | org.apache.kafka.consumer.background.event.queue.size | int | The current number of events in the consumer background event queue | kafka.consumer:type=consumer-metrics,client-id={clientId} |
Compatibility, Deprecation, and Migration Plan
...