Status

Current State: Accepted.

Discussion Thread: https://lists.apache.org/thread/4o5lcl2nbzt0sf28rzx90rnyyf50xfx2

JIRA: KAFKA-20088 - Getting issue details... STATUS

Motivation

Managing Kafka ACLs in large environments is painful: Kafka only allows specifying exact IP addresses or a wildcard (*) for host-based access control. With 50 clients in a subnet, this means 50 separate ACL entries. Not fun. This becomes especially frustrating when the goal is simply to say "allow anything from 10.0.0.0/8" like every other system does.

Two prior KIPs tried to solve this:

• KIP-252 proposed CIDR and IP range support but went stale without addressing metadata version gating.
• KIP-753 focused on CLI changes but didn't fully specify IPv6 or version gating.

This KIP picks up where they left off with:

1. Proper metadata version gating for safe cluster-wide rollout
2. Full IPv4 and IPv6 CIDR support because it is 2026, and we shoul not ignore IPv6
3. A practical implementation path using Apache Commons Net with IPv6 support

Public Interfaces

What Changes for Users

The --allow-host and --deny-host options in kafka-acls.sh will accept CIDR notation alongside the existing formats:

Admin Client API

No changes to the Admin interface or CreateAclsResult. CIDR notation goes in the host field of AccessControlEntry exactly as an exact IP would.

Metadata Version Gating

We add a new metadata version to gate this feature. This is important because older brokers will not understand CIDR patterns, and we need to prevent users from creating ACLs that would break things during a rolling upgrade.

Attempting to create a CIDR ACL before the cluster is ready results in a clear error:

org.apache.kafka.common.errors.UnsupportedVersionException:
CIDR-based ACL host patterns require metadata version IBP_4_X_IVZ or higher.
Current cluster metadata version: IBP_4_0_IV0

Note that didMetadataChange is set to true for IBP_4_4_IV0. This means that downgrading below this version is automatically classified as an unsafe metadata downgrade.

Proposed Changes

1. Host Matching Logic

Today, StandardAuthorizerData.java does simple string comparison and we would extend this to handle CIDR. 

2. Validation Rules

When creating ACLs, the MetadataVersion is passed into AclControlManager.createAcls() (following the same pattern as DelegationTokenControlManager) and threaded through to validateNewAcl(), which calls validateHostPattern():

This ensures that:

• (i.) IPv4 prefix length is 0-32 (enforced by SubnetUtils);
• (ii.) IPv6 prefix length is 0-128 (enforced by SubnetUtils6) and
• (iii.) invalid patterns are rejected with clear error message.

Apache Commons Net and its SubnetUtils class handles IPv4 CIDR matching nicely as well as SubnetUtils6 for IPv6. Moreover, we need to add the dependency (i.e., commons-net) into Kafka.

QuorumController.createAcls() passes featureControl.metadataVersionOrThrow() to AclControlManager.createAcls(), consistent with how other managers such as DelegationTokenControlManager receive the metadata version. 

DENY priority for overlapping CIDR ACLs (IPv4 and IPv6)

The existing Kafka ACL evaluation semantics (i.e., where DENY always takes precedence over ALLOW), regardless of specificity apply unchanged to CIDR-based host patterns for both IPv4 and IPv6. If a client IP matches both an ALLOW CIDR and a DENY CIDR, the request is denied, regardless of prefix length. For example, an ALLOW on 2001:db8::/32 combined with a DENY on 2001:db8:abcd::/48 will deny any client within the /48 range, even though it also falls within the broader /32 ALLOW. So to be even more concrete:

• Client at 2001:db8:abcd::5 matches both ALLOW /32 and DENY /48 results in DENIED (i.e., DENY always wins)
• Client at 2001:db8:ffff::1 matches only ALLOW /32, not the DENY /48 results in ALLOWED

This is consistent with how Kafka already handles overlapping exact-IP ALLOW and DENY entries, and no new priority rules are introduced.

Compatibility

Existing ACLs work exactly as before (i.e., no changes to exact IP or wildcard matching). The CIDR matching logic is additive and only activates when an ACL host pattern contains a '/' character. 

Mixed-version clusters

CIDR ACLs cannot be created until all brokers support them. This is enforced at ACL creation time via MetadataVersion. Any attempt to create a CIDR-based ACL on an older metadata version results in an UnsupportedVersionException with the specific version required. This guarantees that CIDR host patterns never appear in the metadata log on clusters where brokers may not understand them.

Downgrades safety 

Since CIDR host patterns are persisted in metadata records, IBP_4_4_IV0 is defined with didMetadataChange = true. This means any downgrade below IBP_4_4_IV0 is automatically treated as an unsafe metadata downgrade. No additional pre-downgrade validation hook is needed.

IPv4-mapped IPv6 address

Java’s networking stack automatically resolves IPv4-mapped IPv6 addresses (e.g., ::ffff:192.168.0.5) to their native IPv4 form (192.168.0.5). As a result, the host address seen by the authorizer is always the plain IPv4 address. Administrators should use IPv4 CIDR notation (e.g., 192.168.0.0/24) for IPv4 subnets and native IPv6 CIDR notation (e.g., 2001:db8::/32) for IPv6 subnets.

Creating ACLs using IPv4-mapped CIDR patterns is not supported:

• (i.) ::ffff:192.168.0.0/24 (RFC 4291 IPv4-mapped form): the JVM normalizes ::ffff:x.x.x.x to a plain IPv4 address, but the CIDR string still contains the ::ffff: prefix which is not valid IPv4 notation. This is rejected at ACL creation time with an InvalidRequestException.
• (ii.) ::192.168.0.0/24 (deprecated RFC 4291 §2.5.5.1 IPv4-compatible form): this is interpreted as a native IPv6 address (0:0:0:0:0:0:c0a8:0) and passes validation as a regular IPv6 CIDR. However, a /24 on a 128-bit IPv6 address masks the top 24 bits, covering 0:: through 0:ff:ffff:ffff:ffff:ffff:ffff:ffff, which is an enormous and almost certainly unintended range. It would match IPv6 clients within that range (e.g., ::1) but would not match plain IPv4 clients (e.g., 192.168.1.5) since IPv4 clients are always evaluated against IPv4 CIDR rules, not IPv6.

Test plan

All related stuff within subnet handling is tested via commons-net library via RFC examples for IPv6 and IPv4 (i.e., https://datatracker.ietf.org/doc/html/rfc5952, https://datatracker.ietf.org/doc/html/rfc1519). In AclControlManagerTest, we will cover validation of host patterns (i.e., valid/invalid IPv4 or IPv6 CIDR, null/empty inputs and even malformed prefixes) and metadata version gating (i.e., CIDR rejected on older version, accepted on IBP_x_x_IVx, end-to-end ACL creation with CIDR hosts and backwards compatibility with exact IPs and wildcards on older versions. Moreover, in FeatureControlManagerTest we will cover pre-downgrade validation mechanism i.e, block metadata version downgreade when validator returns an error, allows when downgrade pass. Also in StandardAuthorizerTest, CIDR host matching for IPv4 and IPv6 (boundary addresses, range membership, invalid/null patterns), full authorizer flow iterating all addresses in a /24 and /120 range, and overlaping CIDR ACLs semantics which would confirm DENY always takes precedence regardless of prefix length. The downgrade lifecycle will be tested in AclControlManagerTest (i.e., creating CIDR ACL (on supported version), attempting and failing to downgrade (to un-supported version), removing ACL, then succesfully downgrading).

Rejected Alternatives

IP Range Notation (e.g., 10.0.0.1-10.0.0.100) KIP-252 proposed supporting arbitrary IP ranges. We are not doing this because CIDR covers the vast majority of real-world use cases, and adding range support would complicate the implementation without much benefit.

Custom Implementation from Scratch We considered writing all the IP parsing and matching code ourselves, but there's no reason to reinvent the wheel when Apache Commons Net already handles IPv4 well. We just need to add IPv6 support.

Pre-downgrade validator hook We initially implemented a preDowngradeValidator callback in FeatureControlManager that QuorumController would register to check for existing CIDR ACLs before allowing a metadata version downgrade. This was replaced by setting didMetadataChange = true on IBP_4_4_IV0, which gives the same protection through the existing unsafe-downgrade mechanism without introducing bespoke validation infrastructure.