Current state: Adopted (2.2.0)
Discussion thread: TBD
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Today Kafka Streams embedded a few lower-level producer, consumer, and admin clients inside itself:
- A consumer client per-thread.
- A restore consumer client per-thread.
- A shared admin client per-instance.
- A producer client per-thread if EOS is turn off; otherwise a producer client per-task.
This KIP proposes to expose these embedded client's ids via the ThreadMetadata. Those clientIds are useful in a number of ways:
- KafkaStreams#metrics() includes all the metrics from its embedded clients, and are organized by MetricName's group (producer, consumer, admin) and tags (clientIds); knowing the clientIds helps to quickly find the corresponding metric from the map.
- When some of the threads have failed due to unexpected error, their embedded clients may also shutdown and be notifying the users; knowing the ids helps with such trouble shooting scenarios.
- Correlated to KIP-345, exposing the consumer client's id is useful for managing static consumer members in operations like scale-in.
Compatibility, Deprecation, and Migration Plan
- This KIP only adds a few util functions into ThreadMetadata, and hence has no impact on compatibility.