Status

Current state: "Under Discussion"

Discussion thread: here

Vote thread: here

JIRA: KAFKA-19931

Motivation

Currently, we use the Admin client to alter configurations through Admin.incrementalAlterConfigs  method. There are two ways to connect to the Kafka cluster: using bootstrap.controllers or bootstrap.servers. With these different configurations, requests are sent to either the controllers or brokers respectively. However, there are behavioral inconsistencies in how controllers and brokers process alter config requests:

  • Null value handling:
    • Brokers only allow null values with the AlterConfigOp.OpType.DELETE  operation type; otherwise, they throw an InvalidRequestException .
    • Controllers allow users to set null values with AlterConfigOp.OpType.DELETE without restrictions. Additionally, when using the controller path, AlterConfigOp.OpType.SET also allows null values, which may result in unintentional deletion of user configurations.
  • Duplicate configuration handling:
    • Brokers disallow users from setting duplicate configurations for the same resource and throw an InvalidRequestException .
    • Controllers allow this behavior without throwing any exception. If the same config appears multiple times within a single request, the later value overwrites the earlier value.
  • Unknown resource type error codes:
    • Brokers throw an INVALID_REQUEST error code when users input an UNKNOWN resource type
    • Controllers throw an UNSUPPORTED_VERSION error code in the same scenario.
  • Invalid dynamic configs:
    • Brokers return an INVALID_REQUEST error code when users provide an invalid value for a dynamic config.
    • Controllers initially accept the request without returning any error, but silently drop the invalid configs during application. As a result, previously valid dynamic configs may also be removed.

These behavioral differences in the incrementalAlterConfigs API between brokers and controllers may confuse both users and Kafka developers. Therefore, we should align this behavior by standardizing on the broker's validation logic for the following reasons:

  • User familiarity:  Most users and applications connect using  bootstrap.servers and are already accustomed to the broker's behavior. 
  • Stricter validation logic: The broker enforces stricter validation rules. This stricter validation helps catch errors early and prevents ambiguous configuration states. Moving from strict to lenient would introduce correctness and safety risks, whereas the current inconsistency is a bug that should be fixed.
  • Clearer error semantics: The broker uses the INVALID_REQUEST error code, which accurately reflects the situation. The controller's use of UNSUPPORTED_VERSION is misleading since this is not a version compatibility issue.

Benefits of alignment:

  • We can unify the processing logic and reduce code complexity
  • Users will have a better experience when using the Admin API
  • Reduced confusion for both users and Kafka developers

Furthermore, some dynamic configuration validations on the broker side depend on the broker's static configuration context. For example, when a broker processes an IncrementalAlterConfigs request, it can cross-reference the proposed dynamic value against its own server.properties settings to detect conflicts or constraint violations. The controller, however, has no visibility into any broker's static configurations. Without this context, the controller cannot replicate the broker's validation logic and instead silently drops invalid configurations during application.

To fully align the controller's validation behavior with the broker's, the controller must have access to the relevant static configurations from each broker. This KIP therefore introduces a mechanism for brokers to report their static (non-sensitive) configurations to the controller during registration. With this information available, the controller can perform the same context-aware validation that brokers already apply, ensuring consistent error handling regardless of whether the request is routed through bootstrap.servers or bootstrap.controllers.

Relationship with KIP-1294

Note on relationship with KIP-1294: The broker static config reporting mechanism (StaticConfigs in BrokerRegistrationRequest and RegisterBrokerRecord) was originally proposed as part of KIP-1294: Metadata-Version-Aware Configuration Constraints. However, we have moved it into this KIP because static config visibility is a direct prerequisite for achieving the validation alignment that this KIP addresses. Specifically, the controller's inability to validate dynamic configs (the fourth inconsistency listed above) stems from the fact that certain broker-side validations depend on static config context that the controller simply does not have today. Without introducing static config reporting here, this KIP cannot fully close the gap between broker and controller validation behavior — the controller would still lack the context needed to replicate what brokers do locally. Since this KIP is a targeted bug fix with immediate operational impact, whereas KIP-1294 introduces a broader metadata-version-aware validation framework that is still under discussion, it is more appropriate to land the static config reporting infrastructure here first. KIP-1294 can then build upon this foundation to implement its metadata-version-aware features without needing to re-introduce the same protocol changes.

Public Interfaces

BrokerRegistrationRequest

We bump BrokerRegistrationRequest to version 6 to include a collection of static configurations. This allows brokers to report their local server.properties settings to the controller during registration.

// Version 6 adds StaticConfigs for broker static config reporting.
{
  "apiKey": 62,
  "type": "request",
  "listeners": ["controller"],
  "name": "BrokerRegistrationRequest",
  "validVersions": "0-6",
  "flexibleVersions": "0+",
  "fields": [
    // ... existing fields unchanged ...
    { "name": "StaticConfigs", "type": "[]StaticConfig", "versions": "6+",
      "about": "Static configs from the broker's server.properties that are required for validation.", "fields": [
      { "name": "Name", "type": "string", "versions": "6+",
        "about": "The config name." },
      { "name": "Value", "type": "string", "versions": "6+", "nullableVersions": "6+",
        "about": "The config value." }
    ]}
  ]
}

BrokerRegistrationResponse

BrokerRegistrationResponse is bumped to version 6 to match the request. No new fields are added to the response.

// Version 6 is the same as version 5 (new field in request).
{
  "apiKey": 62,
  "type": "response",
  "name": "BrokerRegistrationResponse",
  "validVersions": "0-6",
  "flexibleVersions": "0+",
  "fields": [
    { "name": "ThrottleTimeMs", "type": "int32", "versions": "0+",
      "about": "Duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota." },
    { "name": "ErrorCode", "type": "int16", "versions": "0+",
      "about": "The error code, or 0 if there was no error." },
    { "name": "BrokerEpoch", "type": "int64", "versions": "0+", "default": "-1",
      "about": "The broker's assigned epoch, or -1 if none was assigned." }
  ]
}

RegisterBrokerRecord

We bump RegisterBrokerRecord to version 5. This version introduces a new tagged field StaticConfigs, allowing the controller to persist the broker's reported static settings in the metadata log.

// Version 5 adds StaticConfigs for broker static config reporting.
{
  "apiKey": 0,
  "type": "metadata",
  "name": "RegisterBrokerRecord",
  "validVersions": "0-5",
  "flexibleVersions": "0+",
  "fields": [
    // ... existing fields unchanged ...
    { "name": "StaticConfigs", "type": "[]BrokerStaticConfig", "versions": "5+",
      "taggedVersions": "5+", "tag": 2,
      "about": "Static configs from the broker's server.properties that are required for validation.", "fields": [
      { "name": "Name", "type": "string", "versions": "5+",
        "about": "The config name." },
      { "name": "Value", "type": "string", "versions": "5+", "nullableVersions": "5+",
        "about": "The config value." }
    ]}
  ]
}

Proposed Changes

We align the controller behavior with the broker behavior by extracting common validation logic and applying it to both code paths. The following describes how each inconsistency is addressed:

Null value handling

  • Validate that null values are allowed only with the AlterConfigOp.OpType.DELETE operation.

  • Throw an InvalidRequestException if null values are used with other operation types (e.g., SET).

  • Apply this validation to both the broker and controller paths before request processing.

Duplicate configuration handling

  • Detect duplicate config keys within a single request and immediately throw an InvalidRequestException.

  • This prevents later values from silently overwriting earlier values on the controller path.

Unknown resource type error codes

  • Validate that the resource type is not UNKNOWN before processing the request.

  • The controller returns INVALID_REQUEST instead of UNSUPPORTED_VERSION for unknown resource types.

Invalid dynamic configs

  • Validate configuration values on the controller before applying them.

  • Throw an InvalidRequestException immediately when validation fails, preventing silent drops of invalid configs.

Broker Static Config Reporting

To enable the controller to perform the same context-aware validation as brokers, we introduce a mechanism for brokers to report their static configurations during registration:

Broker side: During registration, the BrokerLifecycleManager collects the static (non-sensitive) configuration entries from server.properties that are required for dynamic config validation. These are included in the StaticConfigs field of BrokerRegistrationRequest version 6.

Controller side: Upon receiving the registration request, the controller persists the reported static configs in the RegisterBrokerRecord (version 5) and maintains them in memory. When processing IncrementalAlterConfigs requests, the controller uses these static configs as validation context, enabling it to perform the same cross-referencing that brokers do locally.

Scope of reported configs: Only the static configurations that are needed as context for dynamic config validation are reported. Sensitive configurations (e.g., passwords, credentials) are never included in the payload.

The validation logic is extracted into a shared utility that is reused by both KafkaApis (broker path) and ControllerApis (controller path), ensuring consistent and deterministic behavior across both code paths.

Compatibility, Deprecation, and Migration Plan

Validation Behavior Alignment

The validation inconsistencies addressed in this KIP (null value handling, duplicate configuration handling, unknown resource type error codes, and invalid dynamic configs) are bug fixes rather than intentional design choices. Therefore, we do not maintain backward compatibility for these cases. The controller will immediately adopt the stricter broker-side validation logic. Since the alter config process does not affect server runtime or startup behavior, this breaking change should have minimal impact on existing deployments.

Static Config Reporting

The introduction of StaticConfigs in BrokerRegistrationRequest version 6 and RegisterBrokerRecord version 5 is a backward-compatible, additive change that does not affect existing deployments:

  • Older brokers (versions < 6) will continue to send BrokerRegistrationRequest at their current version without the StaticConfigs field. The controller will accept these requests normally. In this case, the controller will not have static config context for those brokers and will perform validation using only the information available in the request itself, which is equivalent to the current behavior.
  • Newer brokers registering with a controller that has been upgraded will include StaticConfigs in their registration payload. The controller will use this context to perform richer cross-referencing during IncrementalAlterConfigs validation, matching what brokers do locally.

Test Plan

All remaining tests should pass, and new unit and integration tests have been added to ensure the new behavior works as expected.

Rejected Alternatives

Print warning message and align behaviour in Kafka 5.0

When users use bootstrap.controllers with Admin.incrementalAlterConfigs, if the request violates the expected behavior, we could print a warning in the server-side log and update the logic in Kafka 5.0.

However, since this is a bug fix rather than a feature change, I don't think we should use this gradual approach.

  • No labels