Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Telemetry metrics

...

The following new client metrics should be added:

Metric Name

Type

Group

Tags

Description

JMX Bean

Telemetry metric name (KIP-714)

last-poll-seconds-ago

Gauge

consumer-share

-consumer

-metrics

client-id 

The number of seconds since the last poll() invocation.

kafka.consumer:type=consumer-share

-consumer

-metrics,name=last-poll-seconds-ago,client-id=([-.\w]+) 

org.apache.kafka.consumer.share.last.poll.seconds.ago 

time-between-poll-avg

Meter

consumer-share

-consumer

-metrics

client-id 

The average delay between invocations of poll() in milliseconds.

kafka.consumer:type=consumer-share

-consumer

-metrics,name=time-between-poll-avg,client-id=([-.\w]+) 

o.a.k.consumer.share.time.between.poll.avg 

time-between-poll-max

Meter

consumer-share-

consumer-

metrics

client-id 

The maximum delay between invocations of poll() in milliseconds.

kafka.consumer:type=consumer-share

-consumer

-metrics,name=last-poll-seconds-ago,client-id=([-.\w]+) 

o.a.k.consumer.share.time.between.poll.max 

poll-idle-ratio-avg

Meter

consumer-share

-consumer

-metrics

client-id 

The average fraction of time the consumer's poll() is idle as opposed to waiting for the user code to process records.

kafka.consumer:type=consumer-share

-consumer

-metrics,name=poll-idle-ratio-avg,client-id=([-.\w]+) 

o.a.k.consumer.share.poll.idle.ratio.avg 

heartbeat-response-time-max

Meter

consumer-share

-consumer

-coordinator-metrics

client-id 

The maximum time taken to receive a response to a heartbeat request in milliseconds.

kafka.consumer:type=consumer-share

-consumer

-coordinator-metrics,name=heartbeat-response-time-max,client-id=([-.\w]+) 

o.a.k.consumer.share.coordinator.heartbeat.response.time.max 

heartbeat-rate

Meter

consumer-share

-consumer

-coordinator-metrics

client-id 

The number of heartbeats per second.

kafka.consumer:type=consumer-share

-consumer

-coordinator-metrics,name=heartbeat-rate,client-id=([-.\w]+) 

o.a.k.consumer.share.coordinator.heartbeat.rate 

heartbeat-total

Meter

consumer-share

-consumer

-coordinator-metrics

client-id 

The total number of heartbeats.

kafka.consumer:type=consumer-share

-consumer

-coordinator-metrics,name=heartbeat-total,client-id=([-.\w]+) 

o.a.k.consumer.share.coordinator.heartbeat.total 

last-heartbeat-seconds-ago

Gauge

consumer-share

-consumer

-coordinator-metrics

client-id 

The number of seconds since the last coordinator heartbeat was sent.

kafka.consumer:type=consumer-share

-consumer

-coordinator-metrics,name=last-heartbeat-seconds-ago,client-id=([-.\w]+) 

o.a.k.consumer.share.coordinator.last.heartbeat.seconds.ago 

rebalance-total

Meter

consumer-share

-consumer

-coordinator-metrics

client-id 

The total number of rebalance events.

kafka.consumer:type=consumer-share

-consumer

-coordinator-metrics,name=rebalance-total,client-id=([-.\w]+) 

o.a.k.consumer.share.coordinator.rebalance.total 

rebalance-rate-per-hour

Meter

consumer-share

-consumer

-coordinator-metrics

client-id 

The number of rebalance events per hour.

kafka.consumer:type=consumer-share

-consumer

-coordinator-metrics,name=rebalance-rate-per-hour,client-id=([-.\w]+) 

o.a.k.consumer.share.coordinator.rebalance.rate.per.hour 

fetch-size-avg

Meter

consumer-share-

consumer-

fetch-manager-metrics

client-id 

The average number of bytes fetched per request.

kafka.consumer:type=consumer-share

-consumer

-fetch-manager-metrics,name=fetch-size-avg,client-id=([-.\w]+) 

o.a.k.consumer.share.fetch.manager.fetch.size.avg 

fetch-size-max

Meter

consumer-share

-consumer

-fetch-manager-metrics

client-id 

The maximum number of bytes fetched per request.

kafka.consumer:type=consumer-share-

consumer-

fetch-manager-metrics,name=fetch-size-max,client-id=([-.\w]+) 

o.a.k.consumer.share.fetch.manager.fetch.size.max 

bytes-fetched-rate

Meter

consumer-share

-consumer

-fetch-manager-metrics

client-id 

The average number of bytes fetched per second.

kafka.consumer:type=consumer-share-

consumer-

fetch-manager-metrics,name=bytes-fetched-rate,client-id=([-.\w]+) 

o.a.k.consumer.share.fetch.manager.bytes.fetched.rate 

bytes-fetched-total

Meter

consumer-share-

consumer-

fetch-manager-metrics

client-id 

The total number of bytes fetched.

kafka.consumer:type=consumer-share-

consumer-

fetch-manager-metrics,name=bytes-fetched-total,client-id=([-.\w]+) 

o.a.k.consumer.share.fetch.manager.bytes.fetched.total 

records-per-request-avg

Meter

consumer-share-

consumer-

fetch-manager-metrics

client-id 

The average number of records in each request.

kafka.consumer:type=consumer-share

-consumer

-fetch-manager-metrics,name=records-per-request-avg,client-id=([-.\w]+) 

o.a.k.consumer.share.fetch.manager.records.per.request.avg 

records-per-request-max

Meter

consumer-share

-consumer

-fetch-manager-metrics

client-id 

The maximum number of records in a request.

kafka.consumer:type=consumer-share-

consumer-

fetch-manager-metrics,name=records-per-request-max,client-id=([-.\w]+) 

o.a.k.consumer.share.fetch.manager.records.per.request.max 

records-fetched-rate

Meter

consumer-share-

consumer-

fetch-manager-metrics

client-id 

The average number of records fetched per second.

kafka.consumer:type=consumer-share

-consumer

-fetch-manager-metrics,name=records-fetched-rate,client-id=([-.\w]+) 

o.a.k.consumer.share.fetch.manager.records.fetched.rate 

records-fetched-total

Meter

consumer-share-

consumer-

fetch-manager-metrics

client-id 

The total number of records fetched.

kafka.consumer:type=consumer-share

-consumer

-fetch-manager-metrics,name=records-fetched-total,client-id=([-.\w]+) 

o.a.k.consumer.share.fetch.manager.records.fetched.total 

acknowledgements-send-rate

Meter

consumer-share

-consumer

-fetch-manager-metrics

client-id 

The average number of record acknowledgements sent per second.

kafka.consumer:type=consumer-share-

consumer-

fetch-manager-metrics,name=acknowledgements-send-rate,client-id=([-.\w]+) 

o.a.k.consumer.share.fetch.manager.acknowledgements.send.rate 

acknowledgements-send-total

Meter

consumer-share

-consumer

-fetch-manager-metrics

client-id 

The total number of record acknowledgements sent.

kafka.consumer:type=consumer-share-

consumer-

fetch-manager-metrics,name=acknowledgements-send-total,client-id=([-.\w]+) 

o.a.k.consumer.share.fetch.manager.acknowledgements.send.total 

acknowledgements-error-rate

Meter

consumer-share

-consumer

-fetch-manager-metrics

client-id 

The average number of record acknowledgements that resulted in errors per second.

kafka.consumer:type=consumer-share-

consumer-

fetch-manager-metrics,name=acknowledgements-error-rate,client-id=([-.\w]+) 

o.a.k.consumer.share.fetch.manager.acknowledgements.error.rate 

acknowledgements-error-total

Meter

consumer-share

-consumer

-fetch-manager-metrics

client-id 

The total number of record acknowledgements that resulted in errors.

kafka.consumer:type=consumer-share

-consumer

-fetch-manager-metrics,name=acknowledgements-error-total,client-id=([-.\w]+) 

o.a.k.consumer.share.fetch.manager.acknowledgements.error.total 

fetch-latency-avg

Meter

consumer-share

-consumer

-fetch-manager-metrics

client-id 

The average time taken for a fetch request.

kafka.consumer:type=consumer-share

-consumer

-fetch-manager-metrics,name=fetch-latency-avg,client-id=([-.\w]+) 

o.a.k.consumer.share.fetch.manager.fetch.latency.avg 

fetch-latency-max

Meter

consumer-share

-consumer

-fetch-manager-metrics

client-id 

The maximum time taken for any fetch request.

kafka.consumer:type=consumer-share

-consumer

-fetch-manager-metrics,name=fetch-latency-max,client-id=([-.\w]+) 

o.a.k.consumer.share.fetch.manager.fetch.latency.max 

fetch-rate

Meter

consumer-share

-consumer

-fetch-manager-metrics

client-id 

The number of fetch requests per second.

kafka.consumer:type=consumer-share

-consumer

-fetch-manager-metrics,name=fetch-rate,client-id=([-.\w]+) 

o.a.k.consumer.share.fetch.manager.fetch.rate 

fetch-total

Meter

consumer-share

-consumer

-fetch-manager-metrics

client-id 

The total number of fetch requests.

kafka.consumer:type=consumer-share

-consumer

-fetch-manager-metrics,name=fetch-total,client-id=([-.\w]+) 

o.a.k.consumer.share.fetch.manager.fetch.total 

fetch-throttle-time-avg

Meter

consumer-share

-consumer

-fetch-manager-metrics

client-id 

The average throttle time in milliseconds.

kafka.consumer:type=consumer-share

-consumer

-fetch-manager-metrics,name=fetch-throttle-time-avg,client-id=([-.\w]+) 

o.a.k.consumer.share.fetch.manager.fetch.throttle.time.avg 

fetch-throttle-time-max

Meter

consumer-share

-consumer

-fetch-manager-metrics

client-id 

The maximum throttle time in milliseconds.

kafka.consumer:type=consumer-share

-consumer

-fetch-manager-metrics,name=fetch-throttle-time-max,client-id=([-.\w]+) 

o.a.k.consumer.share.fetch.manager.fetch.throttle.time.max 

Future Work

There are some obvious extensions to this idea which are not included in this KIP in order to keep the scope manageable.

...