...

1. /config/users/<user>/clients/<client-id>
2. /config/users/<user>
3. /config/users/<default>/clients/<client-id>
4. /config/users/<default>/clients/<default>
5. /config/users/<default>
6. /config/clients/<client-id>
7. /config/clients/<default>

Quota Entity

Quotas are currently configured for client-ids. All clients with the same client-id are currently grouped together as a quota entity, enforcing one quota for all clients with the same client-id. This KIP proposes to define quotas for safe client groups which share the same user-principal and client-id and user-level quotas that limit the total quota of users in addition to the existing client-id quotas.

...

• Default quota for <user, client-id> will be stored in Zookeeper at /config/users/<default>/clients/<default>.
• Default user quota will be stored in Zookeeper at the top level config for /config/users/<default>.
• Default client-id quota will be stored in Zookeeper at the top level config for /config/clients/<default>. If not specified, the broker properties quota.producer.default, quota.consumer.default will be used as the default client-id quota for clients of users with unlimited quota.

...

1. If quota override is defined for <userN, clientX> in /config/users/userN/clients/clientX, this quota is allocated for the sole use of <userN, clientX>.
2. If user quota override is defined for userN, clientX in /config/users/userN shares this quota with other clients of userN
3. If <user, client-id> quota is defined in /config/users/<default>clients/clientX, this quota is allocated for the sole use of <userN, clientX>
4. If default <user, client-id> quota is defined in /config/users/<default>clients/<default>, this quota is allocated for the sole use of <userN, clientX>
5. If default user quota is defined in /config/users/<default>, clientX shares this default quota with other clients of userN
6. If client-id quota override is defined for clientX in /config/clients/clientX, this quota is shared across client-id <clientX> of all users
   
7. If dynamic client-id default is configured in /config/clients/<default>, this default quota is shared across client-id <clientX> of all users
8. If quota.producer.default is configured for the broker in server.properties, this is shared across client-id <clientX> of all users
9. Client is not throttled

...

// Default user quota
// Zookeeper persistence path /config/users/<default>
{
    "version":1,
    "config": {
        "producer_byte_rate":"10000",
        "consumer_byte_rate":"20000"
    }
}

// Quotas for user1 (without client-id overrides).
// Zookeeper persistence path /config/users/<encoded-user1>
{
    "version":1,
    "config": {
        "producer_byte_rate":"1024",
        "consumer_byte_rate":"2048"
    }
}

// TopUser-level total quotas for user2
// Zookeeper persistence path /config/users/<encoded-user2>
{
    "version":1,
    "config": {
        "producer_byte_rate":"4096",
        "consumer_byte_rate":"8192"
    }
}
// Quota override for <user2, clientA>
// Zookeeper persistence path /config/users/<encoded-user2>/clients/clientA
{
    "version":1,
    "config": {
        "producer_byte_rate":"10",
        "consumer_byte_rate":"30"
    }
}
// Quota override for <user2, clientB>
// Zookeeper persistence path /config/users/<encoded-user2>/clients/clientB
{
    "version":1,
    "config": {
        "producer_byte_rate":"20",
        "consumer_byte_rate":"40"
    }
} 

// Quotas for client-id clientA of users without user quota override.
// Zookeeper persistence path /config/clients/clientA
{
    "version":1,
    "config": {
        "producer_byte_rate":"100",
        "consumer_byte_rate":"200"
    }
}

...

Client-id based quota configuration overrides will continue be stored under /config/clients, but these will be applied only to clients of users without a quota override and only if default user quota is unlimited. Quota configuration overrides for user principals will be stored under /config/users/<user>. Note that url-encoded version of the user principal will be used as node name under /config/users to cope with Zookeeper naming restrictions. Default user quotas will be stored under /config/users/<default>. Quota overrides for clients of a user will be stored under /config/users/<user>/clients.

...

// Change notification for default user quota
{
    "version":2,
    "entity_path": "users/<default>"
}
// Change notification for user quota of user1
{
    "version":2,
    "entity_path": "users/user1"
}
// Change notification for quota of <user2, clientA>
{
    "version":2,
    "entity_path": "users/user2/clients/clientA"
}
// Change notification for default client-id quota
{
    "version":2,
    "entity_path": "clients/<default>"
}
// Change notification for client-id quota of clientA
{
    "version":2,
    "entity_path": "clients/clientA"
}

...

kafka-configs.sh will be extended to support a new entity type "users". Quota configuration for users will be provided as key-value pairs to be consistent with other configuration options. Hence no new command line arguments will be added to the tool. The tool will parse the key-value pairs specifying rate limits, validate these and convert them to the equivalent JSON for persistence in Zookeeper. The existing entity “clients” will continue to be supported to set client-id quotas for users with unlimited quota. The tool will be extended to accept multiple entity types to configure <user, client-id> quotas. The tool will also be updated to configure default quotas at the top-level quotas  (/config/users/<default>, /config/clients/<default>, /config/users/<default>/clients/<default>).

Compatibility, Deprecation, and Migration Plan

...