Status

Current state: Accepted

Discussion thread: https://lists.apache.org/thread/m6m1qng5v9q8dd1mng0sygc9x83mx8kn

Voting thread: https://lists.apache.org/thread/w1myl1hsmd5bdko6y8y2fdtzvtcsfh13

JIRA: KAFKA-19740 - Getting issue details... STATUS

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Following KIP-1022: Formatting and Updating Features, the group.coordinator.rebalance.protocols server configuration has become redundant. KIP-848 (New Group Coordinator), KIP-932 (Share Group), and KIP-1071 (Streams rebalance protocol) now implement group.version , streams.version , and share.version  respectively. These feature versions enable cluster-wide upgrades and downgrades through kafka-features.sh, eliminating the need for server-level configuration to control feature versions.

Having a server-level configuration is problematic because individual brokers can be misconfigured, resulting in a cluster operating in a mixed state. Additionally, there is inconsistent behavior across coordinator types: while share groups have completely decoupled from group.coordinator.rebalance.protocols, the new group coordinator and streams still depend on this configuration. This means that even when the feature version is upgraded via kafka-features.sh, the new capabilities remain unavailable unless the server configuration is also updated. This dual requirement exists because the feature flag mechanism was introduced later in the development cycle, resulting in a transitional period where both systems coexist. This can lead to user confusion when feature upgrades don't immediately enable new capabilities.

Public Interfaces

  • org.apache.kafka.coordinator.group.GroupCoordinatorConfig.GROUP_COORDINATOR_REBALANCE_PROTOCOLS_CONFIG

Proposed Changes

The group.coordinator.rebalance.protocols configuration will be deprecated in Kafka 4.3 and removed in version 5.0. Both New Group Coordinator and Streams will follow the Share Groups model, where functionality is controlled solely by group.version  and streams.version features via kafka-features.sh. During the deprecation period, if users are missing any of default values "classic, consumer, streams", a warning will inform them that this config will be deprecated and they cannot disable a protocol with it in version 5.0.

This eliminates the dual requirement of setting both configuration and feature flags, providing a consistent experience across all coordinator types. Users should use kafka-features.sh for all feature version management and can safely remove this configuration from their server.properties files, as it no longer affects feature functionality.

Compatibility, Deprecation, and Migration Plan

Current Status (Kafka 4.2 - Upcoming Release)

Share Groups has already deprecated and decoupled from this configuration (though still listed in valid values in the config description). 
Both Group Coordinator and Streams rebalance protocol are currently controlled by both the group.coordinator.rebalance.protocols  configuration and their respective feature flags (`group.version` and `streams.version`).

Deprecation Timeline

Kafka 4.3 (Deprecation Period):

As described in the Proposed Changes, if users explicitly set group.coordinator.rebalance.protocols  with values missing any of the defaults (classic, consumer, streams), users can still disable protocols, but will receive a deprecation warning advising them to either:
- Use the default values (classic, consumer, streams), or
- Remove the configuration

Kafka 5.0 (Removal):
This configuration will be completely removed. Any protocols previously disabled via this configuration will become available, as protocol control will be managed exclusively through feature flags.

Test Plan

This change will be validated through unit and integration tests

Rejected Alternatives


References

  • No labels