Child pages
  • Kafka data structures in Zookeeper
Skip to end of metadata
Go to start of metadata


1. Topic registration info:
/brokers/topics/[topic] :

2. Partition state info:

3. Broker registration info:

4. Controller epoch: 

/controller_epoch -> int (epoch)

5. Controller registration:

/controller -> int (broker id of the controller)

6. Consumer registration:


7. Consumer owner:

/consumers/[groupId]/owners/[topic]/[partitionId] -> string (consumerId)

8. Consumer offset:

/consumers/[groupId]/offsets/[topic]/[partitionId] -> long (offset)

9. Re-assign partitions


10. Preferred replication election


11. Delete topics
/admin/delete_topics/[topic_to_be_deleted] (the value of the path in empty)


Topic Configuration



/config/changes/[config_change_x] -> "topic_name"

Contains the name of the topic that changed.


Client and Topic configuration overrides: The content of both znodes has the same structure



Config Change notification (Topic and Client config)
ISR Change notification


Gets created when ISR is changed at any broker, controller watches for these notifications and sends MetadataUpdateRequest to all brokers.

Broker registration info


ACL info. The content of these znodes have the same structure





Broker registration info


  • No labels


  1. Can we cover the singleton entries too? It would be good to have a single document that covers all the zk entries and standardizes those names too.

    Some comments:

    • The partition assignment uses the partition as a string, it should be an int.
    • I suspect leaderAndIsr is not the best name, since if you add a third thing X it needs to be renamed to leaderAndIsrAndX. Maybe /brokers/topics/test/partitions/partition-id/replicaInfo
    • I think underscores in json are more standard than camel case
    • We should figure out whether we are doing paths with underscores, dashes, or camel case
    1. Maybe instead of singleton values, we wrap the value with a simple single-element JSON structure like {"what_this_is": x}

      So for offset management, instead of '/consumers/my-group/offsets/some-topic/0' -> '42' we have '/consumers/my-group/offsets/some-topic/0' -> '{"offset": 42}'

  2. Thanks for the comments. Made another pass:

    • standardized on underscores for both path and values
    • changed leaderAndIsr to state
    • added all paths there including singleton values.

    Things that I didn't change according to the comments:

    • The partition assignment still uses the partition as a string since in JSON, map keys can only be strings.
    • I left all singleton values as they are, instead of wrapping them in JSON. I prefer to keep things simple. If we need to put more values there, we likely need a new path since the original path name may no longer be appropriate.
  3. Perhaps /controllerEpoch should change to /controller_epoch.

  4. We also need to include the zookeeper path format changes for the following -

    1. Reassigned partitions path
    2. Preferred replica election path