DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
Status
Current state: Accepted
Discussion thread: here
JIRA: KAFKA-19734
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Motivation
Building upon the improvements introduced in KIP-1091: Improved Kafka Streams operator metrics, operators have gained valuable visibility into Kafka Streams applications through enhanced metrics. Currently, the client-state metric provides the running status of the state of an individual Kafka Streams client within an application, as well as the unique processId generated when starting up Kafka Streams. In a single-instance deployment scenario, it's trivial to identify the individual KafkaSteams instance. But in practice, it's a common deployment practice to run several instances on the same server or distributed across different machines. In Kafka Steams, multiple applications with the same applicationId are considered part of a single logical instance, with the processId differentiating the separate physical instances. But the processId is a UUID, which makes it impossible to tie together the different members of the same logical application.
The point of this KIP is to add another JMX tag to the client state metric representing the user provided applicationId providing operators and developers the means to group logical Kafka Streams instances together.
Public Interfaces
New Metric Label
The following table shows the client-state metric with the addition of the application-id in the metric tags.
| Name | Recording Level | Metric Type | Description | Group | Tags | Telemetry Name | MBean Name |
|---|---|---|---|---|---|---|---|
| client-state | INFO | Gauge | The current state of the Kafka Streams instance | stream-metrics | client-id, process-id, application-id | org.apache.kafka.stream.client.state | kafka.streams:type=stream-metrics,client-id=([-.\w]+), process-id=([-.\w]+) |
`
Proposed Change
The applicationId will be available in every client-state metric. Since there is only one client-state per deployed application, the additional label should not present a burden on brokers.
Compatibility, Deprecation, and Migration Plan
- No impact is expected for existing users, as no existing interfaces or behavior is being changed.
- The expected impact of adding a single tag to a single metric will be negligible.
Test Plan
Existing metrics tests will be updated to support these new metrics.
Rejected Alternatives
It was discussed that the client-id contains the application-id thus providing developers and operators a way to group logical instances. But that will only be the case when users have not provided a value for it. In those cases where the client.id is provided, Kafka Streams favors the configured value over concatenating one together with the application-id leaving users in the same position as before. For this reason, this approach was abandoned.