This Confluence has been LDAP enabled, if you are an ASF Committer, please use your LDAP Credentials to login. Any problems file an INFRA jira ticket please.

Child pages
  • Kafka Improvement Proposals
475 more child pages

Versions Compared


  • This line was added.
  • This line was removed.
  • Formatting was changed.


KIP-1 - Remove support of request.required.acks0.9.0.0
KIP-2 - Refactor brokers to allow listening on multiple ports and IPs0.9.0.0
KIP-3 - Mirror Maker Enhancement0.9.0.0
KIP-4 - Command line and centralized administrative operations0.9.0.0,,
KIP-4 - Metadata Protocol Changes0.10.0.0
KIP-8 - Add a flush method to the producer API0.9.0.0
KIP-11 - Kafka Authorizer design0.9.0.0
KIP-12 - Kafka Sasl/Kerberos and SSL implementation0.9.0.0
KIP-13 - Quota Design0.9.0.0
KIP-15 - Add a close method with a timeout in the producer0.9.0.0
KIP-16 - Automated Replica Lag Tuning0.9.0.0
KIP-19 - Add a request timeout to NetworkClient0.9.0.0
KIP-20 Enable log preallocate to improve consume performance under windows and some old Linux file system0.9.0.0
KIP-21 - Dynamic Configuration0.9.0.0 (WIP)
KIP-22 - Expose a Partitioner interface in the new producer0.9.0.0
KIP-25 - System test improvements0.9.0.0
KIP-26 - Add Kafka Connect framework for data import/export0.9.0.0
KIP-28 - Add a processor client0.10.0.0
KIP-31 - Move to relative offsets in compressed message sets0.10.0.0
KIP-32 - Add timestamps to Kafka message0.10.0.0
KIP-33 - Add a time based log index0.10.1.0
KIP-35 - Retrieving protocol version0.10.0.0
KIP-36 - Rack aware replica assignment0.10.0.0
KIP-38: ZooKeeper Authentication0.9.0.0
KIP-40: ListGroups and DescribeGroup0.9.0.0
KIP-41: Consumer Max Records0.10.0.0
KIP-42: Add Producer and Consumer Interceptors0.10.0.0
KIP-43: Kafka SASL enhancements0.10.0.0
KIP-45 - Standardize all client sequence interaction on j.u.Collection.
KIP-48 Delegation token support for Kafka1.0.0 (WIP)
KIP-50 - Move Authorizer to o.a.k.common package0.10.1.0
KIP-51 - List Connectors REST API0.10.0.0
KIP-52: Connector Control APIs0.10.0.0
KIP-54: Sticky Partition Assignment Strategy0.11.0.0
KIP-55: Secure Quotas for Authenticated Users0.10.1.0
KIP-56: Allow cross origin HTTP requests on all HTTP methods0.10.0.0
KIP-57 - Interoperable LZ4 Framing0.10.0.0
KIP-58 - Make Log Compaction Point Configurable0.10.1.0
KIP-60 - Make Java client classloading more flexible0.10.1.0
KIP-62: Allow consumer to send heartbeats from a background thread0.10.1.0
KIP-63: Unify store and downstream caching in streams0.10.1.0
KIP-65: Expose timestamps to Connect0.10.1.0
KIP-66: Single Message Transforms for Kafka Connect0.10.2.0 /
KIP-67: Queryable state for Kafka Streams0.10.1.0
KIP-70: Revise Partition Assignment Semantics on New Consumer's Subscription Change0.10.1.0
KIP-71: Enable log compaction and deletion to co-exist0.10.1.0
KIP-72: Allow putting a bound on memory consumed by Incoming request 1.0.0
KIP-73: Replication Quotas0.10.1.0
KIP-74: Add Fetch Response Size Limit in Bytes0.10.1.0
KIP-75 - Add per-connector Converters0.10.1.0
KIP-78: Cluster Id0.10.1.0
KIP-79 - ListOffsetRequest/ListOffsetResponse v1 and add timestamp search methods to the new consumer0.10.1.0
KIP-77: Improve Kafka Streams Join Semantics0.10.2.0
KIP-81: Bound Fetch memory usage in the consumer1.1.0 (WIP)
KIP-82 - Add Record Headers0.11.0.0
KIP-84: Support SASL SCRAM mechanisms0.10.2.0
KIP-85: Dynamic JAAS configuration for Kafka clients0.10.2.0
KIP-88: OffsetFetch Protocol Update0.10.2.0
KIP-89: Allow sink connectors to decouple flush and offset commit0.10.2.0
KIP-90 - Remove zkClient dependency from Streams0.10.2.0
KIP-91 Provide Intuitive User Timeouts in The Producer1.1.0 (WIP)
KIP-92 - Add per partition lag metrics to KafkaConsumer0.10.2.0
KIP-93: Improve invalid timestamp handling in Kafka Streams0.10.2.0
KIP-94 Session Windows0.10.2.0
KIP-96 - Add per partition metrics for in-sync and assigned replica count0.10.2.0
KIP-97: Improved Kafka Client RPC Compatibility Policy0.10.2.0
KIP-98 - Exactly Once Delivery and Transactional Messaging0.11.0.0
KIP-99: Add Global Tables to Kafka Streams0.10.2.0
KIP-100 - Relax Type constraints in Kafka Streams API0.10.2.0
KIP-101 - Alter Replication Protocol to use Leader Epoch rather than High Watermark for Truncation0.11.0.0
KIP-102 - Add close with timeout for consumers0.10.2.0
KIP-103: Separation of Internal and External traffic0.10.2.0
KIP-104: Granular Sensors for Streams
KIP-105: Addition of Record Level for Sensors0.10.2.0

KIP-106 - Change Default unclean.leader.election.enabled from True to False
KIP-107: Add purgeDataBefore() API in AdminClient0.11.0.0
KIP-108: Create Topic Policy0.10.2.0
KIP-109: Old Consumer Deprecation0.11.0.0
KIP-112: Handle disk failure for JBOD1.0.0
KIP-113: Support replicas movement between log directories1.1.0
KIP-114: KTable state stores and improved semantics0.11.0.0
KIP-115: Enforce offsets.topic.replication.factor upon __consumer_offsets auto topic creation0.11.0.0
KIP-117: Add a public AdminClient API for Kafka admin operations0.11.0.0
KIP-118: Drop Support for Java 7 in Kafka (WIP)
KIP-119: Drop Support for Scala 2.10 in Kafka
KIP-120: Cleanup Kafka Streams builder API1.0.0
KIP-121: Add KStream peek method

KIP-122: Add Reset Consumer Group Offsets tooling0.11.0.0
KIP-123: Allow per stream/table timestamp extractor0.11.0.0
KIP-124 - Request rate quotas0.11.0.0
KIP-126 - Allow KafkaProducer to split and resend oversized batches.
KIP-128: Add ByteArrayConverter for Kafka Connect0.11.0.0
KIP-129: Streams Exactly-Once Semantics0.11.0.0
KIP 130: Expose states of active tasks to KafkaStreams public API1.0.0
KIP-133: Describe and Alter Configs Admin APIs0.11.0.0
KIP-134: Delay initial consumer group rebalance0.11.0.0
KIP-136: Add Listener name to SelectorMetrics tags0.11.0.0
KIP-137: Enhance TopicCommand --describe to show topics marked for deletion0.11.0.0
KIP-138: Change punctuate semantics1.0.0
KIP-140: Add administrative RPCs for adding, deleting, and listing ACLs0.11.0.0
KIP-143: Controller Health Metrics0.11.0.0
KIP-144: Exponential backoff for broker reconnect attempts0.11.0.0
KIP-146 - Classloading Isolation in Connect

KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner1.1.0 (WIP)
KIP-150 - Kafka-Streams Cogroup1.1.0 (WIP)
KIP-151 Expose Connector type in REST API0.11.0.0
KIP-152 - Improve diagnostics for SASL authentication failures1.0.0
KIP-153: Include only client traffic in BytesOutPerSec metric0.11.0.0
KIP-154 Add Kafka Connect configuration properties for creating internal topics0.11.0.0
KIP 155 - Add range scan for windowed state stores0.11.0.0
KIP 156 Add option "dry run" to Streams application reset tool0.11.0.0
KIP 157 - Add consumer config options to streams reset tool1.0.0
KIP-160: Augment KStream.print(), KStream.writeAsText() to allow users pass in extra parameters in the printed string1.0.0
KIP-161: streams deserialization exception handlers1.0.0
KIP-162: Enable topic deletion by default1.0.0
KIP-163: Lower the Minimum Required ACL Permission of OffsetFetch1.0.0

KIP-164 - Add UnderMinIsrPartitionCount and per-partition UnderMinIsr metrics

KIP-167: Add interface for the state store restoration process1.0.0
KIP-168: Add GlobalTopicCount and GlobalPartitionCount metric per cluster1.0.0
KIP-171 - Extend Consumer Group Reset Offset for Stream Application1.1.0
KIP-173: Add prefix to StreamsConfig to enable setting default internal topic configs1.0.0
KIP-175: Additional '--describe' views for ConsumerGroupCommand1.1.0
KIP-177: Consumer perf tool should count rebalance time1.0.0
KIP-180: Add a broker metric specifying the number of consumer group rebalances in progress1.1.0
KIP-182: Reduce Streams DSL overloads and allow easier use of custom storage engines1.0.0
KIP-183 - Change PreferredReplicaLeaderElectionCommand to use AdminClient1.1.0 (WIP)
KIP-187 - Add cumulative count metric for all Kafka rate metrics1.0.0
KIP-188 - Add new metrics to support health checks1.0.0
KIP-189: Improve principal builder interface and add support for SASL1.0.0
KIP-190: Handle client-ids consistently between clients and brokers1.0.0
KIP-191: KafkaConsumer.subscribe() overload that takes just Pattern1.0.0
KIP-192 : Provide cleaner semantics when idempotence is enabled1.0.0
KIP-195: AdminClient.createPartitions1.0.0
KIP-196: Add metrics to Kafka Connect framework1.0.0
KIP-197 Connect REST API should include the connector type when describing a connector1.0.0
KIP-198: Remove ZK dependency from Streams Reset Tool1.0.0
KIP-202: Move merge() from StreamsBuilder to KStream1.0.0
KIP-203: Add toLowerCase support to rule 1.1.0
KIP-204 : Adding records deletion operation to the new Admin Client API1.1.0
KIP-205: Add all() and range() API to ReadOnlyWindowStore1.1.0
KIP-207: Offsets returned by ListOffsetsResponse should be monotonically increasing even during a partition leader change1.1.0 (WIP)
KIP-210 - Provide for custom error handling when Kafka Streams fails to produce1.1.0
KIP-214: Add config to the broker1.1.0 (WIP)
KIP-215: Add topic regex support for Connect sinks1.1.0 (WIP)
KIP-218: Make KafkaFuture.Function java 8 lambda compatible1.1.0 (WIP)
KIP-220: Add AdminClient into Kafka Streams' ClientSupplier1.1.0
KIP-223 - Add per-topic min lead and per-partition lead metrics to KafkaConsumer1.1.0
KIP-224: Add configuration parameter `retries` to Streams API1.1.0
KIP-225 - Use tags for consumer “records.lag” metrics1.1.0 (WIP)
KIP-238: Expose Kafka cluster ID in Connect REST API1.1.0 (WIP)
KIP-239 Add queryableStoreName() to GlobalKTable1.1.0 (WIP)

KIPs under discussion

KIP-59: Proposal for a kafka broker commandSent emails to Dev discussion group. Work tracked under KAFKA-3663.
KIP-86: Configurable SASL callback handlers 
KIP-110: Add Codec for ZStandard Compression 
KIP-125: ZookeeperConsumerConnector to KafkaConsumer Migration and Rollback 
KIP-131 - Add access to OffsetStorageReader from SourceConnector 
KIP-135 : Send of null key to a compacted topic should throw non-retriable error back to user 
KIP 141 - ProducerRecord & SourceRecord: Add timestamp constructors 
KIP-142: Add ListTopicsRequest to efficiently list all the topics in a cluster 
KIP-145 - Expose Record Headers in Kafka Connectbuilds on KIP-82 (would be ideal to make 0.11 release so it can go out as one, but not mandatory)
KIP-148: Add a connect timeout for client 
KIP-158: Kafka Connect should allow source connectors to set topic-specific settings for new topics 
KIP-159: Introducing Rich functions to Streams 
KIP-165: Extend Interactive Queries for return latest update timestamp per key


KIP-166 - Add a tool to make amounts of replicas and leaders on brokers balanced 
KIP-169 - Lag-Aware Partition Assignment Strategy 

KIP-174 - Deprecate and remove internal converter configs in WorkerConfig

KIP-176: Remove deprecated new-consumer option for tools 
KIP-178: Size-based log directory selection strategy


KIP-185: Make exactly once in order delivery the default producer setting

KIP-186: Increase offsets retention default to 7 days 
KIP-193: Add SchemaBuilder.from(Schema) 
KIP-199: Add Kafka Connect offset tool 
KIP-201: Rationalising Policy interfaces 
KIP-206: Add support for UUID serialization and deserializationVoting in progress
KIP-208: Add SSL support to Kafka Connect REST interface


KIP-209: Connection String Support 
KIP-211: Revise Expiration Semantics of Consumer Group Offsets 
KIP-212: Enforce set of legal characters for connector names 
KIP-213 Support non-key joining in KTable 
KIP-216: IQ should throw different exceptions for different errors 
KIP-217: Expose a timeout to allow an expired ZK session to be re-created 

KIP-219 - Improve quota communication

KIP-221: Repartition Topic Hints in Streams 
KIP-222 - Add Consumer Group operations to Admin APIVoting in progress
KIP-226 - Dynamic Broker Configuration 
KIP-227 - Introduce Incremental FetchRequests to Increase Partition Scalability 
KIP-228 Negative record timestamp support 
KIP-229: DeleteGroups API 
KIP-231: Improve the Required ACL of ListGroups APIVoting in progress
KIP-232: Detect outdated metadata using per-partition leaderEpoch field 
KIP-233: Simplify StreamsBuilder#addGlobalStore 
KIP-234: add support for getting topic defaults from AdminClient 
KIP-235: Add DNS alias support for secured connection 
KIP-236: Interruptible Partition Reassignment 
KIP-237: More Controller Health Metrics 
KIP-239 Add queryableStoreName() to GlobalKTable KIP-240: AdminClient.listReassignments() AdminClient.describeReassignments() 
KIP-243: Make ProducerConfig and ConsumerConfig constructors public   
KIP-244: Add Record Header support to Kafka Streams 
KIP-245: Use Properties instead of StreamsConfig in KafkaStreams constructorVoting in progress