Note that this KIP proposal is derived from a proposal by Grant Henke that was part of KIP-4 - Command line and centralized administrative operations.
Current state: Under Discussion
Discussion thread: TBD
JIRA:
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
KIP-4 - Command line and centralized administrative operations outlines the motivation for exposing admin operations via the protocol:
A few specific use cases worth pointing out:
A new ACL resource type `Configs` will be introduced with Describe, Alter and All operations. Initially, this resource type will work at a global level like the Cluster resource type. That is, the only valid resource name will be "kafka-cluster". This leaves the door open for evolving this to be more fine-grained in the future.
Introduce 2 new ACL operations: ReadConfigs and WriteConfigs.
KIP-140: Add administrative RPCs for adding, deleting, and listing ACLs introduced a wire format representation for ResourceType and AclOperation. We weill add new values to both types as follows:
AclOperation
0: Unknown
1: Any
2: All
3: Read
4: Write
5: Create
6: Delete
7: Alter
8: Describe
9: ClusterAction
10: ReadConfigs (new)
11: WriteConfigs (new)
ResourceType
0: Unknown
1: Any
2: Topic
3: Group
4: Cluster
5: Broker (new)
6: Client (new)
7: User (new)
ListConfigs Request (Version: 0) => [resource] resource => resource_type resource_name resource_type => INT8 resource_name => STRING |
Request semantics:
ListConfigs Response (Version: 0) => error_code [entities] entities => error_code resource_type resource_name [configs] error_code => INT16 resource_type => INT8 resource_name => STRING configs => config_name => STRING config_value => STRING read_only => BOOLEAN is_default => BOOLEAN is_sensitive => BOOLEAN |
AlterConfigs Request (Version: 0) => [resources] validate_only validate_only => BOOLEAN resources => resource_type resource_name [configs] resource_type => INT8 resource_name => STRING configs => config_name => STRING config_value => STRING |
Request Semantics
Alter
operation is attempted on a read-only config, an UnsupportedOperation
error will be returned for the relevant resource.AlterConfigs Response (Version: 0) => [responses] responses => resource_type resource_name error_code error_message resource_type => INT8 resource_name => STRING error_code => INT16 error_message => STRING |
Policy
In a similar fashion to KIP-108: Create Topic Policy, we allow users to define a policy class to validate alter configs requests. The config name will be alter.configs.policy.class.name
and the interface follows:
package org.apache.kafka.server.policy; public interface AlterConfigsPolicy extends Configurable, AutoCloseable { /** * Class containing the create request parameters. */ class RequestMetadata { /** * Create an instance of this class with the provided parameters. * * This constructor is public to make testing of <code>AlterConfigPolicy</code> implementations easier. */ public RequestMetadata(ConfigResource resource, Map<String, String> configs) { ... } /** * Return configs in the request. */ public Map<String, String> configs() { ... } } /** * Validate the request parameters and throw a <code>PolicyViolationException</code> with a suitable error * message if the alter configs request parameters for the provided resource do not satisfy this policy. * * Clients will receive the POLICY_VIOLATION error code along with the exception's message. Note that validation * failure only affects the relevant resource, other resources in the request will still be processed. * * @param requestMetadata the alter configs request parameters for the provided resource. * @throws PolicyViolationException if the request parameters do not satisfy this policy. */ void validate(RequestMetadata requestMetadata) throws PolicyViolationException; } |
Users will have to ensure that the policy implementation code is in the broker's classpath. Implementations should throw the existing PolicyViolationException
with an appropriate error message if the request does not fulfill the policy requirements. We chose a generic name for the only parameter of the validate
method in case we decide to add parameters that are not strictly related to the topic (e.g. session information) in the future. The constructor of RequestMetadata
is public to make testing convenient for users. Under normal circumstances, it should only be instantiated by Kafka. We chose to create separate API classes instead of reusing request classes to make it easier to evolve the latter.
They follow a similar pattern as existing AdminClient APIs:
public class AdminClient { public ListConfigsResult listConfigs(Collection<ConfigResource> resources, ListConfigsOptions options); public AlterConfigsResult alterConfigs(Map<ConfigResource, Config> configs, AlterConfigsOptions options); } public class ListConfigsOptions { public ListConfigsOptions timeoutMs(Integer timeout); } public class ListConfigsResult { public Map<ConfigResource, KafkaFuture<Config>> results() public KafkaFuture<Map<ConfigResource, Config>> all(); } public class AlterConfigsOptions { public AlterConfigsOptions timeoutMs(Integer timeout); public AlterConfigsOptions validateOnly(boolean validateOnly); } public class AlterConfigsResult { public KafkaFuture<Void> all(); public Map<ConfigResource, KafkaFuture<Void>> results(); } public class ConfigResource { enum Type { CLIENT, BROKER, TOPIC, USER, UNKNOWN; } public ConfigResource(Type type, String name) { ... } public Type type() { ... } public String name() { ... } } public class Config { public Config(Collection<ConfigEntry> configs) { ... } public Collection<ConfigEntry> entries() { ... } } public class ConfigEntry { public ConfigEntry(String name, String value) { this(name, value, false, false, false); } public ConfigEntry(String name, String value, boolean isDefault, boolean isSensitive, boolean isReadOnly) { ... } public String name() { ... } public String value() { ... } public boolean isDefault { ... } public boolean isSensitive { ... } public boolean isReadOnly { ... } } |
Proposed Changes
The "Public Interfaces" section covers all the proposed changes.
We are adding new ACL operations, so third-party Authorizer implementations will potentially have to be updated in order to support them. Since this only affects newly introduced protocol and AdminClient APIs, it's not a compatibility issue. It simply means that the new functionality won't be available for users of such Authorizer implementations until they are updated.
With regards to forwards compatibility, ConfigResource.Type has an UNKNOWN element in case it receives data from a newer broker that cannot be mapped to one of the existing enum types.