This Confluence has been LDAP enabled, if you are an ASF Committer, please use your LDAP Credentials to login. Any problems file an INFRA jira ticket please.

Child pages
  • Restful API Proposal

Versions Compared


  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migrated to Confluence 4.0

I am working on a proposal to satisfy the Restful Proxy project listed on the Projects page. (

My working knowledge of Kafka is about 3 hours old and I have actually written zero code yet, it is my job to do so for Box so I will update this as my knowledge increases. There may be some fundamental issues with this proposal at this time.

General Idea

Create a Restful API which will allow producers and consumers to communicate with a Kafka server via HTTP calls


URL Format:

GET /messages/<topic>/<consumer>/<offset> - Retrieve the messages for a consumer, from the topic, starting with an offset

DELETE /messages/<topic>/<consumer>/<offset> - ACK consumption of message for a consumer, topic, offset


PUT /messages/<topic>

Request Body will become the message

Known Missing Items:

  • I know that this does not take into account partitions.
  • It would be nice to be able to implement the async producer and have an HTTP callback happen from within the RESTServer (producer specifies a URL to be requested as callback)
  • If an HTTP call back wasn't available, have the service return a task ID (MD5 of the message or something), and then allow the client to query the server to see if the callback from Kafka has happened yet.