Status

Current State: Under Discussion

Discussion Thread: https://lists.apache.org/thread/4o5lcl2nbzt0sf28rzx90rnyyf50xfx2

JIRA: KAFKA-20088 - Getting issue details... STATUS

Motivation

Managing Kafka ACLs in large environments is painful: Kafka only allows specifying exact IP addresses or a wildcard (*) for host-based access control. With 50 clients in a subnet, this means 50 separate ACL entries. Not fun. This becomes especially frustrating when the goal is simply to say "allow anything from 10.0.0.0/8" like every other system does.

Two prior KIPs tried to solve this:

• KIP-252 proposed CIDR and IP range support but went stale without addressing metadata version gating.
• KIP-753 focused on CLI changes but didn't fully specify IPv6 or version gating.

This KIP picks up where they left off with:

1. Proper metadata version gating for safe cluster-wide rollout
2. Full IPv4 and IPv6 CIDR support because it is 2026, and we shoul not ignore IPv6
3. A practical implementation path using Apache Commons Net with IPv6 support

Public Interfaces

What Changes for Users

The --allow-host and --deny-host options in kafka-acls.sh will accept CIDR notation alongside the existing formats:

Admin Client API

For programmatic use via the Admin Client, nothing changes in the API signatures. CIDR notation goes where a host would normally go:

AclBinding binding = new AclBinding(
    new ResourcePattern(ResourceType.TOPIC, "events", PatternType.LITERAL),
    new AccessControlEntry("User:app", "10.0.0.0/8", AclOperation.READ, AclPermissionType.ALLOW)
);
adminClient.createAcls(Collections.singleton(binding));

Metadata Version Gating

Moreover, we propose adding a new metadata version to gate this feature. This is important because older brokers will not understand CIDR patterns, and we need to prevent users from creating ACLs that would break things during a rolling upgrade.

// In MetadataVersion.java
IBP_4_X_IVZ(XX, "4.X", "IVZ", false),

public boolean isCidrAclSupported() {
    return this.isAtLeast(IBP_4_X_IVZ);
}

Attempting to create a CIDR ACL before the cluster is ready results in a clear error:

org.apache.kafka.common.errors.UnsupportedVersionException:
CIDR-based ACL host patterns require metadata version IBP_4_X_IVZ or higher.
Current cluster metadata version: IBP_4_0_IV0

Proposed Changes

1. Host Matching Logic

Today, StandardAuthorizerData.java does simple string comparison:

// Current code
if (!acl.host().equals(WILDCARD) && !acl.host().equals(host)) {
    return null;
}

We extend this to handle CIDR:

// Check if the host matches. If it doesn't, return no result (null).
if (!acl.host().equals(WILDCARD) && !acl.host().equals(host) && !hostMatchesCidr(host, acl.host())) {
    return null;
}

The hostMatchesCidr method handles CIDR matching for both IPv4 and IPv6:

static boolean hostMatchesCidr(String host, String cidrPattern) {
    if (cidrPattern == null || !cidrPattern.contains("/")) {
        return false;
    }

    try {
		InetAddress address = CidrUtils.parseCidrAddress(cidrPattern);
        if (CidrUtils.isIpv6(address)){
			return new SubnetUtils6(cidrPattern).getInfo().isInRange(host);
        } else {
        	SubnetUtils subnet = new SubnetUtils(cidrPattern);
            subnet.setInclusiveHostCount(true);
            return subnet.getInfo().isInRange(host);
        }
	} catch (IllegalArgumentException e) {
    	return false;
	}
}


// parseCidrAddress will strip prefix length i.e., from "192.168.0.0/24" we do "192.168.0.0" before its passed into InetAddress.getByName(...). 

2. Validation Rules

When creating ACLs, we validate CIDR patterns in AclControlManager.java:

/**
 * Validates the host pattern of an ACL entry.
 *
 * Accepts:
 * - Wildcard "*" (matches any host)
 * - Valid IPv4 address (e.g., "192.168.1.1")
 * - Valid IPv6 address (e.g., "2001:db8::1")
 * - Valid IPv4 CIDR notation (e.g., "192.168.0.0/24"), which requires cidrSupported=true
 * - Valid IPv6 CIDR notation (e.g., "2001:db8::/32"), which requires cidrSupported=true
 *
 * @param host The host pattern to validate
 * @param cidrSupported Whether CIDR notation is supported by the current metadata version
 * @throws InvalidRequestException if the host pattern is invalid
 * @throws UnsupportedVersionException if CIDR notation is used but not supported
 */
static void validateHostPattern(String host, boolean cidrSupported) {
    if (host == null || host.isEmpty()) {
        throw new InvalidRequestException("Host pattern cannot be null or empty");
    }

    if ("*".equals(host)) {
        return;
    }

    if (host.contains("/")) {
        if (!cidrSupported) {
            throw new UnsupportedVersionException(
                "CIDR-based ACL host patterns require metadata version " +
                MetadataVersion.IBP_4_X_IVZ + " or higher.");
        }
        validateCidrNotation(host);
    }
}

/**
 * Validates a CIDR notation pattern.
 * Supports both IPv4 (e.g., "192.168.0.0/24") and IPv6 (e.g., "2001:db8::/32") CIDR patterns.
 *
 * @param cidrPattern The CIDR pattern to validate
 * @throws InvalidRequestException if the CIDR pattern is invalid
 */
static void validateCidrNotation(String cidrPattern) {
    try {
    	InetAddress address = CidrUtils.parseCidrAddress(cidrPattern);
        if (CidrUtils.isIpv6(address)) {
			new SubnetUtils6(cidrPattern);
		} else {
			new SubnetUtils(cidrPattern);
		}
	} catch (IllegalArgumentException e)
		throw new InvalidRequestException("Invalid CIDR notation '" + cidrPattern + "': " + e.getMessage());
   }
}

This ensures that (i.) IPv4 prefix length is 0-32 (enforced by SubnetUtils); (ii.) IPv6 prefix length is 0-128 (enforced by SubnetUtils6) and (iii.) invalid patterns are rejected with clear error message.

Apache Commons Net and its SubnetUtils class handles IPv4 CIDR matching nicely as well as SubnetUtils6 for IPv6. Moreover, we need to add the dependency (i.e., commons-net) into Kafka.

DENY priority for overlapping CIDR ACLs (IPv4 and IPv6)

The existing Kafka ACL evaluation semantics (i.e., where DENY always takes precedence over ALLOW), regardless of specificity apply unchanged to CIDR-based host patterns for both IPv4 and IPv6. If a client IP matches both an ALLOW CIDR and a DENY CIDR, the request is denied, regardless of prefix length. For example, an ALLOW on 2001:db8::/32 combined with a DENY on 2001:db8:abcd::/48 will deny any client within the /48 range, even though it also falls within the broader /32 ALLOW. So to be even more concrete:

• Client at 2001:db8:abcd::5 matches both ALLOW /32 and DENY /48 results in DENIED (i.e., DENY always wins)
• Client at 2001:db8:ffff::1 matches only ALLOW /32, not the DENY /48 results in ALLOWED

This is consistent with how Kafka already handles overlapping exact-IP ALLOW and DENY entries, and no new priority rules are introduced.

Compatibility

Existing ACLs work exactly as before (i.e., no changes to exact IP or wildcard matching). The CIDR matching logic is additive and only activates when an ACL host pattern contains a '/' character. 

Mixed-version clusters

CIDR ACLs cannot be created until all brokers supported them. This is enforced at ACL creation time in AclControlManager.validateHostPattern(). Any attempt to create a CIDR-based ACL on an older metadata version results in a an UnsupportedVersionException (where an user will be warned with specific version,  which is required). This guarantees that CIDR host patterns never appear in AccessControlEntryRecord entries on clusters where brokers may not understand them.

Downgrades safety 

Since CIDR host patterns are persisted in AccessControlEntryRecord metadata records, allowing metadata version downgrade while such records exist would leave the cluster in an inconsistent state (i.e., the ACLs would be still stored and replayed, but older brokers would not regnonize the CIDR format and would fail to match correctly. 

So to prevent this we introduce pre-downgrade validation step. Before any metadata version downgrade is applied, FeatureControlManager invokes a validator registered by QuorumController that checks whether CIDR ACLs currently exists. If the target metadata version does not support CIDR and any ACL host pattern contains CIDR notation, the downgrade is rejected with the error:

Cannot downgrade below IBP_x_y_IVz while CIDR-based ACL host patterns exist: [192.168.0.0/24, 2001:db8::/32, ... ]. 
Remove all CIDR ACLs first.

IPv4-mapped IPv6 address

Javas networking stack automatically resolves IPv4-mapped IPv6 addresses (e.g., ::ffff:192.168.0.5) to their native IPv4 form (192.168.0.5). As a result, the host address seen by the authorizer is always the plain IPv4 address. Administrators should use IPv4 CIDR notation (e.g., 192.168.0.0/24) for IPv4 subnets and native IPv6 CIDR notation (e.g., 2001:db8::/32) for IPv6 subnets. Creating ACLs using IPv4-mapped IPv6 CIDR patterns such as ::ffff:192.168.0.0/120 is not supported and will not match IPv4 clients, since the clients resolved address will go through the IPv4 matching path while the ACL pattern would be evaluated as IPv6. 

Test plan

All related stuff within subnet handling is tested via commons-net library via RFC examples for IPv6 and IPv4 (i.e., https://datatracker.ietf.org/doc/html/rfc5952, https://datatracker.ietf.org/doc/html/rfc1519). In AclControlManagerTest, we will cover validation of host patterns (i.e., valid/invalid IPv4 or IPv6 CIDR, null/empty inputs and even malformed prefixes) and metadata version gating (i.e., CIDR rejected on older version, accepted on IBP_x_x_IVx, end-to-end ACL creation with CIDR hosts and backwards compatibility with exact IPs and wildcards on older versions. Moreover, in FeatureControlManagerTest we will cover pre-downgrade validation mechanism i.e, block metadata version downgreade when validator returns an error, allows when downgrade pass. Also in StandardAuthorizerTest, CIDR host matching for IPv4 and IPv6 (boundary addresses, range membership, invalid/null patterns), full authorizer flow iterating all addresses in a /24 and /120 range, and overlaping CIDR ACLs semantics which would confirm DENY always takes precedence regardless of prefix length. The downgrade lifecycle will be tested in AclControlManagerTest (i.e., creating CIDR ACL (on supported version), attempting and failing to downgrade (to un-supported version), removing ACL, then succesfully downgrading).

Rejected Alternatives

IP Range Notation (e.g., 10.0.0.1-10.0.0.100) KIP-252 proposed supporting arbitrary IP ranges. We are not doing this because CIDR covers the vast majority of real-world use cases, and adding range support would complicate the implementation without much benefit.

Custom Implementation from Scratch We considered writing all the IP parsing and matching code ourselves, but there's no reason to reinvent the wheel when Apache Commons Net already handles IPv4 well. We just need to add IPv6 support.