Status

Current state: Under Discussion

Discussion thread: https://lists.apache.org/thread/01bs42crlkn5jr1r846vflp293zv2t86

JIRA: KAFKA-20018

Motivation

The `kafka-feature.sh` command-line tool currently has two issues that impact usability and consistency:

KAFKA-19749

The `--bootstrap-server` /` --bootstrap-controller` arguments are defined as a globally required group. This incorrectly prevents offline subcommands like `version-mapping` and `feature-dependencies` from running without providing unnecessary connection arguments. This behavior is inconsistent with other tools, such as `kafka-storage.sh`, which allows its `version-mapping` command to run without cluster connection arguments.

KAFKA-19544

A recent change introduced a backend flag (`unstableFeatureVersionsEnabled`) to `MetadataVersion.fromVersionString()` to control the inclusion of unstable metadata versions. This capability is not exposed to the user in the `version-mapping` command, making it impossible for users to control the query of the feature mapping for unstable versions.

Proposed Change

Move Cluster Connection Arguments into Subparsers:

The `MutuallyExclusiveGroup` for `--bootstrap-server` and `--bootstrap-controller` will be removed from the top-level parser. Instead, this required argument group will be added individually to each of the subcommands that require a live cluster connection: `describe`, `upgrade`, `downgrade`, and `disable`. This change makes the arguments required only when one of these specific subcommands is chosen, allowing offline commands to run without them.

Breaking Change Warning Message

Since this is a breaking change, it should be introduced in 5.0. A warning message will be added to the helping string in 4.x telling users that the group will be removed from the top level and become arguments under specific commands.

Expose Unstable Version Flag in version-mapping:

A new boolean flag, `--unstable-feature-versions`, will be added to the `version-mapping` subcommand in both `kafka-feature.sh` and `kafka-storage.sh`. When this flag is present, the tool will call `MetadataVersion.fromVersionString()` with the unstableFeatureVersionsEnabled flag set to true, allowing users to see feature mappings for development and testing versions. By default (without the flag), only production/stable metadata versions will be included.


Excluding unstable feature versions is the behavior before KAFKA-19544, so this is not a breaking change and can be introduced in 4.x.

New and Changed Public Interfaces

kafka-feature.sh

The `--bootstrap-server` / `--bootstrap-controller` arguments are no longer valid as top-level arguments. They must now be specified after the subcommand (e.g., `kafka-feature.sh describe --bootstrap-server ...`). 

Scripts invoking offline commands will no longer require these arguments.


A new flag, `--unstable-feature-versions`, will be added to the version-mapping subcommand.

kafka-storage.sh

A new flag, `--unstable-feature-versions`, will be added to the version-mapping subcommand to maintain consistency with `kafka-feature.sh`.

Default Behavior of version-mapping

The default semantic of the version-mapping command in both tools will be changed to exclude unstable feature versions. This aligns with the behavior before KAFKA-19544. Users who need to see the mapping for unstable versions must now explicitly add the --unstable-feature-versions flag.

Migration Plan and Compatibility

This KIP introduces a breaking change to the command-line interface of kafka-feature.sh. Users must update existing scripts that use `kafka-features.sh`.


Users whose workflows rely on version-mapping returning mappings for unstable versions will need to add the new --unstable-feature-versions flag to their scripts.

Rejected Alternatives

Make Top-Level Arguments Optional and Check Programmatically

We considered keeping the arguments at the top level, making them optional (removing required(true)), and then programmatically checking for their presence in the execute logic for online commands. This was rejected, as it makes the tool's interface less explicit and can be misleading to users. By moving the arguments into the subparsers, the tool's built-in --help output for each online subcommand will clearly and automatically show that the bootstrap arguments are required, which is a more user-friendly and self-documenting approach.

  • No labels