Status

Current state: "Under Discussion"

Discussion thread: here

Vote thread: here

JIRA: KAFKA-19931

Motivation

Currently, we use the Admin client to alter configurations through Admin.incrementalAlterConfigs  method. There are two ways to connect to the Kafka cluster: using bootstrap.controllers or bootstrap.servers. With these different configurations, requests are sent to either the controllers or brokers respectively. However, there are behavioral inconsistencies in how controllers and brokers process alter config requests:

• Null value handling:
  • Brokers only allow null values with the AlterConfigOp.OpType.DELETE  operation type; otherwise, they throw an InvalidRequestException .
  • Controllers allow users to set null values with AlterConfigOp.OpType.DELETE without restrictions. Additionally, when using the controller path, AlterConfigOp.OpType.SET also allows null values, which may result in unintentional deletion of user configurations.
• Duplicate configuration handling:
  • Brokers disallow users from setting duplicate configurations for the same resource and throw an InvalidRequestException .
  • Controllers allow this behavior without throwing any exception. If the same config appears multiple times within a single request, the later value overwrites the earlier value.
• Unknown resource type error codes:
  • Brokers throw an INVALID_REQUEST error code when users input an UNKNOWN resource type
  • Controllers throw an UNSUPPORTED_VERSION error code in the same scenario.
• Invalid dynamic configs:
  • Brokers return an INVALID_REQUEST error code when users provide an invalid value for a dynamic config.
  • Controllers initially accept the request without returning any error, but silently drop the invalid configs during application. As a result, previously valid dynamic configs may also be removed.

These behavioral differences in the incrementalAlterConfigs API between brokers and controllers may confuse both users and Kafka developers. Therefore, we should align this behavior by standardizing on the broker's validation logic for the following reasons:

• User familiarity:  Most users and applications connect using  bootstrap.servers and are already accustomed to the broker's behavior. 
• Stricter validation logic: The broker enforces stricter validation rules. This stricter validation helps catch errors early and prevents ambiguous configuration states. Moving from strict to lenient would introduce correctness and safety risks, whereas the current inconsistency is a bug that should be fixed.
• Clearer error semantics: The broker uses the INVALID_REQUEST error code, which accurately reflects the situation. The controller's use of UNSUPPORTED_VERSION is misleading since this is not a version compatibility issue.

Benefits of alignment:

• We can unify the processing logic and reduce code complexity
• Users will have a better experience when using the Admin API
• Reduced confusion for both users and Kafka developers

Public Interfaces

This KIP does not change any public interfaces.

Proposed Changes

We align the controller behavior with the broker behavior by extracting common validation logic and applying it to both code paths. The following describes how each inconsistency is addressed:

Null value handling

• Validate that null values are allowed only with the AlterConfigOp.OpType.DELETE operation.

• Throw an InvalidRequestException if null values are used with other operation types (e.g., SET).

• Apply this validation to both the broker and controller paths before request processing.

Duplicate configuration handling

• Detect duplicate config keys within a single request and immediately throw an InvalidRequestException.

• This prevents later values from silently overwriting earlier values on the controller path.

Unknown resource type error codes

• Validate that the resource type is not UNKNOWN before processing the request.

• The controller returns INVALID_REQUEST instead of UNSUPPORTED_VERSION for unknown resource types.

Invalid dynamic configs

• Validate configuration values on the controller before applying them.

• Throw an InvalidRequestException immediately when validation fails, preventing silent drops of invalid configs.

The validation logic is extracted into a shared utility that is reused by both KafkaApis (broker path) and ControllerApis (controller path), ensuring consistent and deterministic behavior across both code paths.

Compatibility, Deprecation, and Migration Plan

Due to these behavioral inconsistencies, this appears to be a bug rather than an intentional design choice. Therefore, we should not maintain backward compatibility for this case. Additionally, since the alter config process does not affect server runtime or startup behavior, this breaking change should have minimal impact on existing deployments.

Test Plan

All remaining tests should pass, and new unit and integration tests have been added to ensure the new behavior works as expected.

Rejected Alternatives

Print warning message and align behaviour in Kafka 5.0

When users use bootstrap.controllers with Admin.incrementalAlterConfigs, if the request violates the expected behavior, we could print a warning in the server-side log and update the logic in Kafka 5.0.

However, since this is a bug fix rather than a feature change, I don't think we should use this gradual approach.