This is an un-ordered compendium of projects and project ideas.
|Table of Contents|
Java Client Re-write
There are a number of limitations in the current JVM clients outlined in the above link.
Authentication (users) and authentication (permissions) at the topic level.
Provide a quota mechanism to limit throughput and # of requests on a per-topic and per-user basis.
We want to improve the client libraries for the major languages (ruby, python, c++, etc). Some of these don't yet have a 0.8 compatible library available and for others the client is somewhat limited and could be improved.
Now that we have replication it would be possible to implement exactly-once producer semantics.
We have added APIs for clients to manage offsets but we still are storing them in Zookeeper under the covers. This is unfortunately not at all scalable for high commit rates for larger numbers of consumers. There is a proposal for implementing a more scalable store for these.
For people who want to publish Kafka feeds for existing applications that produce log files it might be nice to have something more sophisticated than the console-producer. This would be a process that ran in the background and tailed log directories and read and published formatted messages.