Versions Compared

Old Version 18

changes.mady.by.user Guozhang Wang

Saved on

compared with

New Version Current

changes.mady.by.user Dustin Cote

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

0.8:

Wiki Markup1. Topic registration info:
/brokers/topics/\[topic\] :

Code Block

Schema:
{ "fields" :
    [ {"name": "version", "type": "int", "doc": "version id"},
      {"name": "partitions",
       "type": {"type": "map",
                "values": {"type": "array", "items": "int", "doc": "a list of replica ids"},
                "doc": "a map from partition id to replica list"},
      }
    ]
}
Example:
{
  "version": 1,
  "partitions": {"0": [0, 1, 3] } }
}

...

2.   Partition   state   info:

 /brokers/topics/\[topic\]/partitions/\[partitionId\]/state
 Code Block

Schema:
{ "fields":
    [ {"name": "version", "type": "int", "doc": "version id"},
      {"name": "isr",
       "type": {"type": "array",
                "items": "int",
                "doc": "an array of the id of replicas in isr"}
      },
      {"name": "leader", "type": "int", "doc": "id of the leader replica"},
      {"name": "controller_epoch", "type": "int", "doc": "epoch of the controller that last updated the leader and isr info"},
      {"name": "leader_epoch", "type": "int", "doc": "epoch of the leader"}
    ]
}

Example:
{
  "version": 1,
  "isr": [0,1],
  "leader": 0,
  "controller_epoch": 1,
  "leader_epoch": 0
}

 Wiki Markup3.   Broker   registration   info:

 /brokers/ids/\[brokerId\]
 Code Block

Schema:
{ "fields":
    [ {"name": "version", "type": "int", "doc": "version id"},
      {"name": "host", "type": "string", "doc": "ip address or host name of the broker"},
      {"name": "port", "type": "int", "doc": "port of the broker"},
      {"name": "jmx_port", "type": "int", "doc": "port for jmx"}
    ]
}

Example:
{
  "version": 1,
  "host": "192.168.1.148",
  "port": 9092,
  "jmx_port": 9999
}

...
6. Consumer registration:
 Wiki Markup/consumers/\[groupId\]/ids/\[consumerId\]
 Code Block

Schema:
{ "fields":
    [ {"name": "version", "type": "int", "doc": "version id"},
      {"name": "pattern", "type": "string", "doc": "can be of static, white_list or black_list"},
      {"name": "subscription", "type" : {"type": "map", "values": {"type": "int"},
                                         "doc": "a map from a topic or a wildcard pattern to the number of streams"}      }    ]
}

Example:
A static subscription:
{
  "version": 1,
  "pattern": "static",
  "subscription": {"topic1": 1, "topic2": 2}
}


A whitelist subscription:
{
  "version": 1,
  "pattern": "white_list",
  "subscription": {"abc": 1}}

A blacklist subscription:
{
  "version": 1,
  "pattern": "black_list",
  "subscription": {"abc": 1}}

7. Consumer owner:
 Wiki Markup/consumers/\[groupId\]/owners/\[topic\]/\[partitionId\]  \ ->   string   (consumerId)
8. Consumer offset:unmigrated-wiki-markup
/consumers/\[groupId\]/offsets/\[topic\]/\[partitionId\]  \ ->   long   (offset)
9. Re-assign partitions
/admin/reassign_partitions
 Code Block

{
   "fields":[
      {
         "name":"version",
         "type":"int",
         "doc":"version id"
      },
      {
         "name":"partitions",
         "type":{
            "type":"array",
            "items":{
               "fields":[
                  {
                     "name":"topic",
                     "type":"string",
                     "doc":"topic of the partition to be reassigned"
                  },
                  {
                     "name":"partition",
                     "type":"int",
                     "doc":"the partition to be reassigned"
                  },
                  {
                     "name":"replicas",
                     "type":"array",
                     "items":"int",
                     "doc":"a list of replica ids"
                  }
               ],
            }
            "doc":"an array of partitions to be reassigned to new replicas"
         }
      }
   ]
}

Example: 
{
  "version": 1,
  "partitions": 
     [
        {
            "topic": "Foo",
            "partition": 1,
            "replicas": [0, 1, 3]
        }
     ]             
}

...
/admin/preferred_replica_election
 Code Block

{
   "fields":[
      {
         "name":"version",
         "type":"int",
         "doc":"version id"
      },
      {
         "name":"partitions",
         "type":{
            "type":"array",
            "items":{
               "fields":[
                  {
                     "name":"topic",
                     "type":"string",
                     "doc":"topic of the partition for which preferred replica election should be triggered"
                  },
                  {
                     "name":"partition",
                     "type":"int",
                     "doc":"the partition for which preferred replica election should be triggered"
                  }
               ],
            }
            "doc":"an array of partitions for which preferred replica election should be triggered"
         }
      }
   ]
}

Example:

{
  "version": 1,
  "partitions": 
     [
        {
            "topic": "Foo",
            "partition": 1          
        },
        {
            "topic": "Bar",
            "partition": 0          
        }
     ]             
}


11. Delete topics
 /admin/delete_topics/[topic_to_be_deleted] (the value of the path in empty)
0.8.1
Topic Configuration
/config/topics/[topic_name]
Example
 Code Block
{
  "version": 1,
  "config": {
    "config.a": "x",
    "config.b": "y",
    ...
  }
}

/config/changes/[config_change_x] -> "topic_name"
Contains the name of the topic that changed.
0.9.0
Client and Topic configuration overrides: The content of both znodes has the same structure
/config/clients/[topic_name]
/config/topics/[topic_name]
 Code Block
{
  "version": 1,
  "config": {
    "config.a": "x",
    "config.b": "y",
    ...
  }
}

Config Change notification (Topic and Client config)
 Code Block
{"version" : 1, "entity_type":"topics/clients", "entity_name" : "topic_name/client_id"}

ISR Change notification
/isr_change_notification/isr_change_x
Gets created when ISR is changed at any broker, controller watches for these notifications and sends MetadataUpdateRequest to all brokers.
Broker registration info
/brokers/ids/[brokerId]
 Code Block
Schema:
{ "fields":
    [ {"name": "version", "type": "int", "doc": "version id"},
      {"name": "host", "type": "string", "doc": "ip address or host name of the broker"},
      {"name": "port", "type": "int", "doc": "port of the broker"},
      {"name": "jmx_port", "type": "int", "doc": "port for jmx"}
      {"name": "endpoints", "type": "array", "items": "string", "doc": "endpoints supported by the broker"}
    ]
}

Example:
{
  "version":2,
  "host","localhost",
  "port",9092
  "jmx_port":9999,
  "timestamp":"2233345666",
  "endpoints": ["PLAINTEXT://host1:9092", "SSL://host1:9093"]
}
ACL info. The content of these znodes have the same structure
/kafka-acl/Topic/[topic_name]
/kafka-acl/Cluster/kafka-cluster
/kafka-acl/Group/[groupId]
 Code Block
{"version": 1, "acls": [ { "host":"host1", "permissionType": "Allow","operation": "Read","principal": "User:alice"}]}
0.10
Broker registration info
/brokers/ids/[brokerId]
 Code Block
Schema:
{ "fields":
    [ {"name": "version", "type": "int", "doc": "version id"},
      {"name": "host", "type": "string", "doc": "ip address or host name of the broker"},
      {"name": "port", "type": "int", "doc": "port of the broker"},
      {"name": "jmx_port", "type": "int", "doc": "port for jmx"}
      {"name": "endpoints", "type": "array", "items": "string", "doc": "endpoints supported by the broker"}
      {"name": "rack", "type
 Code Block
 
Schema:
{ "fields": 
    [ {"name": "version", "type": "int", "doc": "version id"},
      {"name": "topics", 
       "type": { "type": "array", "items": "string", "doc": "anRack arrayof ofthe topicsbroker. toOptional. beThis deleted"}will 
be used in rack aware replication } ]
}

Example:
{
  "version": 1,
  "topics": ["foo", "bar"]
}

0.8.1
Topic Configuration
 Wiki Markup
/config/topics/\[topic_name\]
Example
 Code Block
assignment for fault tolerance."} 
    ]
}

Example:
{
  "version": 13,
  "host":"localhost",
  "configport": {9092,
  "jmx_port":9999,
  "config.atimestamp": "x2233345666",
  "endpoints":  "config.b": "y"["PLAINTEXT://host1:9092", "SSL://host1:9093"],
    ...
  }
}

 Wiki Markup
/config/changes/\[config_change_x\] -> "topic_name"
...
"rack": "us-east-1c"
}