DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
Status
Current state: Under Discussion
Discussion thread:
JIRA:
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Motivation
KIP-932 introduced Share Groups to Kafka, enabling cooperative consumption with queueing semantics. Currently, the behavior of share groups is governed by several broker-level configurations. While some configurations can be dynamically updated, some limits are currently static and applied globally to all share groups on a broker. This KIP aims to address this limitation by enabling dynamic, group-level configuration for these essential parameters.
Public Interfaces
The following table summarizes the configuration parameters for Share Groups. It indicates which parameters are currently supported as dynamic group configurations and which are introduced by this KIP.
| Broker Configuration Key | Description | Group Level Dynamic Config Support |
|---|---|---|
| group.share.session.timeout.ms | The timeout to detect client failures when using the share group protocol. | Yes |
| group.share.heartbeat.interval.ms | The heartbeat interval given to the members of a share group. | Yes |
| group.share.record.lock.duration.ms | The record acquisition lock duration in milliseconds for share groups. | Yes |
| group.share.partition.max.record.locks | Share-group record lock limit per share-partition. | No |
| group.share.delivery.count.limit | The maximum number of delivery attempts for a record delivered to a share group. | No |
| group.share.max.size | The maximum number of members that a single share group can accommodate. | No |
Proposed Changes
We propose to extend the dynamic group configuration mechanism to include share.partition.max.record.locks, share.delivery.count.limit, and share.max.size. This allows these parameters to be dynamically configured and tuned for each share groups at runtime.
Compatibility, Deprecation, and Migration Plan
This change is fully backward compatible. Older broker versions will continue to apply only the broker-level static configurations. After upgrading to the new version, users can dynamically update the group-level configurations to override these values.
Test Plan
Unit tests and integration tests will be added.
Rejected Alternatives
None.