Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. Click
    Create from template
    templateName54329345
    templateId54329345
    titleKIP-NEXT: Insert Title Here
    buttonLabelCreate KIP
    . Take the next available KIP number and give your proposal a descriptive heading. e.g. "KIP 42: Allow Infinite Retention With Bounded Disk Usage".
  2. Fill in the sections as described above
  3. Start a [DISCUSS] thread on the Apache mailing list. Please ensure that the subject of the thread is of the format [DISCUSS] KIP-{your KIP number} {your KIP heading} The and the body contains a link to your new KIP. The discussion should happen on the mailing list, not on the wiki, since the wiki comment system doesn't work well for larger discussions. In the process of the discussion you may update the proposal. You should let people know the changes you are making. When you feel you have a finalized proposal 
  4. Once the proposal is finalized call a [VOTE] to have the proposal adopted. These proposals are more serious than code changes and more serious even than release votes. The criteria for acceptance is lazy majority. The vote should remain open for at least 72 hours.
  5. Please update the KIP wiki page, and the index below, to reflect the current stage of the KIP after a vote. This acts as the permanent record indicating the result of the KIP (e.g., Accepted or Rejected). Also report the result of the KIP vote to the voting thread on the mailing list so the conclusion is clear.

KIP round-up

Next KIP Number: 435526

Use this number as the identifier for your KIP and increment this value.

Adopted KIPs

Please insert new rows in sorted order (descending by KIP number).

393: Time windowed serde to properly deserialize changelog input topic2386: Standardize on Min/Avg/Max metrics' default value2380: Detect outdated control requests and bounced brokers using broker generation2377: TopicCommand to use AdminClient2376: Implement AutoClosable on appropriate classes that want to be used in a try-with-resource statement2371: Add a configuration to build custom SSL principal name2 368: Allow SASL Connections to Periodically Re-Authenticate2365: Materialized, Serialized, Joined, Consumed and Produced with implicit Serde1338 Support to exclude the internal topics in kafka-topics.sh command1336: Consolidate ExtendedSerializer/Serializer and ExtendedDeserializer/Deserializer1332: Update AclCommand to use AdminClient API1330 retentionPeriod in SessionBytesStoreSupplier1328: Ability to suppress updates for KTables2.1.0 (partially implemented) / 2.3.0 (WIP)324: Add method to get metrics() in AdminClient1322: Return new error code for DeleteTopics API when topic deletion disabled.1321: Update TopologyDescription to better represent Source and Sink Nodes1320: Allow fetchers to detect and handle log truncation2.1.0 (partially implemented) / 2.2.308: Support dynamic update of max.connections.per.ip/max.connections.per.ip.overrides1298: Error Handling in Connect0289: Improve the default group id behavior in KafkaConsumer-284: Set default retention ms for Streams repartition topics to Long.MAX_VALUE0282: Add the listener name to the authentication context0KIP279: Fix log divergence between leader and follower after fast leader fail over0276 - Add StreamsConfig prefix for different consumers0268 Simplify Rebalance Metadata Upgrade0267: Add Processor Unit Test Support to Kafka Streams Test Utils0265: Make Windowed Serde to public APIs0.0261: Add Single Value Fetch in Window Stores0.0257 - Configurable Quota Management078251: Allow timestamp manipulation in Processor API079249: Add Delegation Token Operations to KafkaAdminClient00247: Add public test utils for Kafka Streams181245: Use Properties instead of StreamsConfig in KafkaStreams constructor0244: Add Record Header support to Kafka Streams Processor 083243: Make ProducerConfig and ConsumerConfig constructors public1.1239 Add queryableStoreName() to GlobalKTable1238: Expose Kafka cluster ID in Connect REST APIKIP-235: Add DNS alias support for secured connection233: Simplify StreamsBuilder#addGlobalStore1891.1227 - Introduce Incremental FetchRequests to Increase Partition Scalability192226 - Dynamic Broker Configuration11218: Make KafkaFuture.Function java 8 lambda compatible1.1100215: Add topic regex support for Connect sinks1101214 zookeeper.max.in.flight.requests config to the broker191: KafkaConsumer.subscribe() overload that takes just Pattern1190: Handle client-ids consistently between clients and brokers1187 - Add cumulative count metric for all Kafka rate metrics12180: Add a broker metric specifying the number of consumer group rebalances in progress1.1176: Remove deprecated new-consumer option for tools173 prefix to StreamsConfig to enable setting default internal topic configs1131163: Lower the Minimum Required ACL Permission of OffsetFetch1162: Enable topic deletion by default.0160: Augment KStream.print(), KStream.writeAsText() to allow users pass in extra parameters in the printed string1.0.0157 - Add consumer config options to streams reset tool0-156 Add option "dry run" to Streams application reset tool0.11.0.0155 - Add range scan for windowed state stores0.11.0-140: Add administrative RPCs for adding, deleting, and listing ACLs0.11.0.0-110: Add Codec for ZStandard Compression1106 - Change Default unclean.leader.election.enabled from True to False0.11.0.070: Revise Partition Assignment Semantics on New Consumer's Subscription Change0.10.145 - Standardize all client sequence interaction on j.u.Collection.0.10.0

KIP (please keep this sorted by KIP number)

Release

1KIP-428: Add in-memory window store482: The Kafka Protocol should Support Optional Tagged Fields2.34.0 (WIP)
2KIP-504 - 425: Add some Log4J Kafka Appender Properties for Producing to Secured BrokersAdd new Java Authorizer Interface2.34.0 (WIP)
3KIP-420503: Add Single Value Fetch in Session Storesmetric for number of topics marked for deletion2.24.0 (WIP)
4KIP-412: Extend Admin API to support dynamic application log levels497: Add inter-broker API to alter ISR2.34.0 (WIP)
5KIP-414: Expose Embedded ClientIds in Kafka Streams496: Administrative API to delete consumer offsets2.24.0 (WIP)
6KIP-402: Improve fairness in SocketServer processors488: Clean up Sum,Count,Total Metrics2.24.0 (WIP)
7KIP-394: Require member.id for initial join group request484: Expose metrics for group and transaction metadata loading duration2.24.0 (WIP)
8KIP-480: Sticky Partitioner2.4.0
9KIP-479: Add Materialized to Join2.4.0(WIP)
10KIP-476: Add Java AdminClient Interface382: MirrorMaker 2.4.02.3.0 (WIP)
11KIP-475: New Metrics to Measure Number of Tasks on a Connector2.4.0 (WIP)
12KIP-474: To deprecate WindowStore#put(key, value)2.4.0 (WIP)
13KIP-471: Expose RocksDB Metrics in Kafka Streams

2.

4.0 (WIP)

14KIP-374: Add '--help' option to all available Kafka CLI commands467: Augment ProduceResponse error messaging for specific culprit records2.24.0 (WIP)
15KIP-372: Naming Repartition Topics for Joins and Grouping465: Add Consolidated Connector Endpoint to Connect REST API2.13.0
16KIP-464: Defaults for AdminClient#createTopic2.4.0
17KIP-462: Use local thread id for KStreams2.3.0
18KIP-367 Introduce close(Duration) to Producer and AdminClient instead of close(long, TimeUnit)461: Improve Replica Fetcher behavior at handling partition failure2.23.0
19KIP-366: Make FunctionConversions deprecated460: Admin Leader Election RPC2.14.0
20KIP-458: Connector Client Config Override Policy2.3.0
21KIPKIP-361: Add Consumer Configuration to Disable Auto Topic Creation455: Create an Administrative API for Replica Reassignment2.34.0 (WIP)
22KIP-359: Verify leader epoch in produce requests454: Expansion of the ConnectClusterState interface2.3.0 (WIP)
23KIP-358: Migrate Streams API to Duration instead of long ms times453: Add close() method to RocksDBConfigSetter2.13.0
24KIP-357449: Add support to list ACLs per principalconnector contexts to log messages in Connect workers 2.13.0
25KIP-356446: Add withCachingDisabled() to StoreBuilderchangelog topic configuration to KTable suppress2.14.0 (WIP)
26KIP-353: Improve Kafka Streams Timestamp Synchronization445: In-memory Session Store2.13.0
27

KIP-

351: Add --under-min-isr option to describe topics command

443: Return to default segment.ms and segment.index.bytes in Streams repartition topics

2.3.0
28KIP-346: Improve LogCleaner behavior on error442: Return to default max poll interval in Streams2.3.02.1
29KIP-342 Add support for custom SASL extensions in OAuthBearer authentication440: Extend Connect Converter to support headers2.14.0 (WIP)
30KIP-341: Update Sticky Assignor's User Data Protocol436: Add a metric indicating start time2.3.0 (WIP)
31KIP-340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file430 - Return Authorized Operations in Describe Responses2.13.0
32KIP-339: Create a new IncrementalAlterConfigs API429: Kafka Consumer Incremental Rebalance Protocol2.34.0 (WIP)
33KIP-428: Add in-memory window store2.3.0
34KIP-427: Add AtMinIsr topic partition category (new metric & TopicCommand option)2.3.0
35KIP-425: Add some Log4J Kafka Appender Properties for Producing to Secured Brokers2.3.0
36KIP-331 Add default implementation to close() and configure() for Serializer, Deserializer and Serde421: Support resolving externalized secrets in AbstractConfig2.3.0 (WIP)
37KIP-420: Add Single Value Fetch in Session Stores2.2.0
38KIP-417: Allow JmxTool to connect to a secured RMI port2.3.0
39KIP-415: Incremental Cooperative Rebalancing in Kafka Connect2.3.0
40KIP-414: Expose Embedded ClientIds in Kafka Streams2.2.0
41KIP-412: Extend Admin API to support dynamic application log levels2.4.0 (WIP)
42

KIP-

411: Make default Kafka Connect worker task client IDs distinct

2.3.0
43KIP-319: Replace segments with segmentInterval in WindowBytesStoreSupplier402: Improve fairness in SocketServer processors2.2.0 (partially implemented) / 2.32.1.0
44KIP-313: Add KStream.flatTransform and KStream.flatTransformValues 401: TransformerSupplier/ProcessorSupplier StateStore connecting2.24.0 (partially implemented)/ 2.3.0 (WIP)
45KIP-312 Add Overloaded StreamsBuilder Build Method to Accept java.util.Properties396: Add Commit/List Offsets Operations to AdminClient2.14.0 (WIP)
46KIP-394: Require member.id for initial join group request2.2.0
47KIP-306: Configuration for Delaying Response to Failed Authentication393: Time windowed serde to properly deserialize changelog input topic2.12.0
48KIP-305: Add Connect primitive number converters389: Introduce a configurable consumer group size limit2.02.0
49KIP-303: Add Dynamic Routing in Streams Sink386: Standardize on Min/Avg/Max metrics' default value2.02.0
50KIP-302 - Enable Kafka clients to use all DNS resolved IP addresses382: MirrorMaker 2.02.14.0
51

KIP-

379: Multiple Consumer Group Management

2.4.0 (WIP)
52KIP-297: Externalizing Secrets for Connect Configurations380: Detect outdated control requests and bounced brokers using broker generation2.02.0
53KIP-295 Add Streams Configuration Allowing for Optional Topology Optimization377: TopicCommand to use AdminClient2.02.0
54KIP-294 - Enable TLS hostname verification by default376: Implement AutoClosable on appropriate classes that want to be used in a try-with-resource statement2.02.0
55KIP-292374: Add transformValues() method to KTable'--help' option to all available Kafka CLI commands2.02.0
56KIP-290372: Support for Prefixed ACLsNaming Repartition Topics for Joins and Grouping2.01.0
57KIP-371: Add a configuration to build custom SSL principal name2.2.0
58KIP-285: Connect Rest Extension Plugin369: Alternative Partitioner to Support "Always Round-Robin" Selection2.04.0
59KIP 368: Allow SASL Connections to Periodically Re-Authenticate2.2.0
60KIP-283: Efficient Memory Usage for Down-Conversion367 Introduce close(Duration) to Producer and AdminClient instead of close(long, TimeUnit)2.02.0
61KIP-366: Make FunctionConversions deprecated2.1.0
62KIP-281: ConsumerPerformance: Increase Polling Loop Timeout and Make It Reachable by the End User365: Materialized, Serialized, Joined, Consumed and Produced with implicit Serde2.01.0
63KIP-361: Add Consumer Configuration to Disable Auto Topic Creation2.3.0
64KIP-278 - Add version option to Kafka's commands360: Improve handling of unknown producer2.04.0 (WIP)
65KIP-277 - Fine Grained ACL for CreateTopics API359: Verify leader epoch in produce requests2.04.0 (WIP)
66KIP-358: Migrate Streams API to Duration instead of long ms times2.1.0
67KIP-274: Kafka Streams Skipped Records Metrics357: Add support to list ACLs per principal2.01.0
68KIP-272356: Add API version tag to broker's RequestsPerSec metricwithCachingDisabled() to StoreBuilder2.01.0
69KIP-270 - A Scala Wrapper Library for Kafka Streams354: Add a Maximum Log Compaction Lag2.03.0
70KIP-353: Improve Kafka Streams Timestamp Synchronization2.1.0
71KIP-352: Distinguish URPs caused by reassignment2.4.0 (WIP)
72KIP-266: Fix consumer indefinite blocking behavior351: Add --under-min-isr option to describe topics command2.03.0
73KIP-346: Improve LogCleaner behavior on error2.1
74KIP-345: Introduce static membership protocol to reduce consumer rebalances2.75KIP-258: Allow to Store Record Timestamps in RocksDB4.0 (WIP), partially available in 2.3.0
7675KIP-342 Add support for custom SASL extensions in OAuthBearer authentication2.1.0
7776KIP-255: OAuth Authentication via SASL/OAUTHBEARER341: Update Sticky Assignor's User Data Protocol2.03.0
77KIP-340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file2.1.0
78KIP-339: Create a new IncrementalAlterConfigs API2.3.
8079KIP-338 Support to exclude the internal topics in kafka-topics.sh command2.1.0
80KIP-336: Consolidate ExtendedSerializer/Serializer and ExtendedDeserializer/Deserializer2.1.0
8281KIP-332: Update AclCommand to use AdminClient API2.1.0
82KIP-331 Add default implementation to close() and configure() for Serializer, Deserializer and Serde2.3.0
8483KIP-330: Add retentionPeriod in SessionBytesStoreSupplier2.1.0
8584KIP-1.1.0
86KIP-237: More Controller Health Metrics2.0.0
328: Ability to suppress updates for KTables2.1.0 (partially implemented) / 2.3.0 (WIP)
85KIP-324: Add method to get metrics() in AdminClient2.1.0
86KIP-322: Return new error code for DeleteTopics API when topic deletion disabled.872.1.0
8887KIP-321: Update TopologyDescription to better represent Source and Sink Nodes2.1.0
88KIP-231: Improve the Required ACL of ListGroups API320: Allow fetchers to detect and handle log truncation2.1.0 90KIP-229: DeleteGroups API(partially implemented) / 2.2.0
9189KIP-319: Replace segments with segmentInterval in WindowBytesStoreSupplier2.1.0
90KIP-1.1.0
93KIP-225 - Use tags for consumer “records.lag” metrics1.1.0
313: Add KStream.flatTransform and KStream.flatTransformValues 

2.2.0 (partially implemented)

/ 2.3.0

91KIP-312 Add Overloaded StreamsBuilder Build Method to Accept java.util.Properties294KIP-224: Add configuration parameter `retries` to Streams API.1.0
9592KIP-223 - Add per-topic min lead and per-partition lead metrics to KafkaConsumer2.0.0
96KIP-222 - Add Consumer Group operations to Admin API2.0.0
308: Support dynamic update of max.connections.per.ip/max.connections.per.ip.overrides2.1.0
93KIP-307: Allow to define custom processor names with KStreams DSL2.3.0 (partial)
94KIP-306: Configuration for Delaying Response to Failed Authentication297KIP-220: Add AdminClient into Kafka Streams' ClientSupplier.1.0
9895KIP-219 - Improve quota communication305: Add Connect primitive number converters2.0.0
9996KIP-303: Add Dynamic Routing in Streams Sink2.0.0
97KIP-302 - Enable Kafka clients to use all DNS resolved IP addresses2.1.0
98KIP-300: Add 1.1.0
102KIP-212: Enforce set of legal characters for connector names1.1.0
103KIP-211: Revise Expiration Semantics of Consumer Group Offsets2.1.0
104KIP-210 - Provide for custom error handling when Kafka Streams fails to produce1.1.0
105KIP-208: Add SSL support to Kafka Connect REST interface1.1.0
106KIP-207: Offsets returned by ListOffsetsResponse should be monotonically increasing even during a partition leader change2.2.0
107KIP-206: Add support for UUID serialization and deserialization2.1.0
108KIP-205: Add all() and range() API to ReadOnlyWindowStore1.1.0
109KIP-204 : Adding records deletion operation to the new Admin Client API1.1.0
110KIP-203: Add toLowerCase support to sasl.kerberos.principal.to.local rule 1.1.0
111KIP-202: Move merge() from StreamsBuilder to KStream1.0.0
112KIP-198: Remove ZK dependency from Streams Reset Tool1.0.0
113KIP-197 Connect REST API should include the connector type when describing a connector1.0.0
114KIP-196: Add metrics to Kafka Connect framework1.0.0
Windowed KTable API in StreamsBuilder2.4.0
99KIP-298: Error Handling in Connect2.0.0
100KIP-297: Externalizing Secrets for Connect Configurations2.0.0
101KIP-295 Add Streams Configuration Allowing for Optional Topology Optimization2.0.0
102KIP-294 - Enable TLS hostname verification by default2.0.0
103

KIP-291: Separating controller connections and requests from the data plane

Accepted
104KIP-292: Add transformValues() method to KTable2.0.0
105KIP-290: Support for Prefixed ACLs2.0.0
106KIP-289: Improve the default group id behavior in KafkaConsumer2.2.0
107KIP-285: Connect Rest Extension Plugin2.0.0
108KIP-284: Set default retention ms for Streams repartition topics to Long.MAX_VALUE2.0.0
109KIP-283: Efficient Memory Usage for Down-Conversion2.0.0
110KIP-282: Add the listener name to the authentication context2.0.0
111KIP-281: ConsumerPerformance: Increase Polling Loop Timeout and Make It Reachable by the End User2.0.0
112KIP-279: Fix log divergence between leader and follower after fast leader fail over2.0.0
113KIP-278 - Add version option to Kafka's commands2.0.0
114KIP-277 - Fine Grained ACL for CreateTopics API2.0.0
115KIP-276 - Add StreamsConfig prefix for different consumers2115KIP-195: AdminClient.createPartitions1.0.0
116KIP-192 : Provide cleaner semantics when idempotence is enabled274: Kafka Streams Skipped Records Metrics21.0.0
117KIP-272: Add API version tag to broker's RequestsPerSec metric2.0.0
118KIP-270 - A Scala Wrapper Library for Kafka Streams2.0.0
119KIP-189: Improve principal builder interface and add support for SASL268: Simplify Kafka Streams Rebalance Metadata Upgrade21.0.0
120KIP-188 - Add new metrics to support health checks267: Add Processor Unit Test Support to Kafka Streams Test Utils21.0.0
121KIP-266: Fix consumer indefinite blocking behavior2.0.0
122KIP-186: Increase offsets retention default to 7 days265: Make Windowed Serde to public APIs2.0.0
123KIP-183 - Change PreferredReplicaLeaderElectionCommand to use AdminClient261: Add Single Value Fetch in Window Stores2.0.0
124KIP-258: Allow to Store Record Timestamps in RocksDB2.3.0 (partially implemented)
124125KIP-182: Reduce Streams DSL overloads and allow easier use of custom storage engines-257 - Configurable Quota Management21.0.0
125126KIP-255: OAuth Authentication via SASL/OAUTHBEARER2.0.0
126127KIP-177: Consumer perf tool should count rebalance time251: Allow timestamp manipulation in Processor API21.0.0
127128KIP-249: Add Delegation Token Operations to KafkaAdminClient2.0.0
128129KIP-175: Additional '--describe' views for ConsumerGroupCommand247: Add public test utils for Kafka Streams1.1.0
129130KIP-174 - Deprecate and remove internal converter configs in WorkerConfig245: Use Properties instead of StreamsConfig in KafkaStreams constructor2.0.0
130131KIP-244: Add Record Header support to Kafka Streams Processor API2.0.0
KIP-171 - Extend Consumer Group Reset Offset for Stream Application1.1.0132KIP-168243: Add GlobalTopicCount and GlobalPartitionCount metric per clusterMake ProducerConfig and ConsumerConfig constructors public1.01.0
133KIP-167: Add interface for the state store restoration process239 Add queryableStoreName() to GlobalKTable1.01.0
134KIP-164 - Add UnderMinIsrPartitionCount and per-partition UnderMinIsr metrics238: Expose Kafka cluster ID in Connect REST API1.01.0
135KIP-237: More Controller Health Metrics2.0.0
136KIP-235: Add DNS alias support for secured connection2.1.0
137KIP-161: streams deserialization exception handlers233: Simplify StreamsBuilder#addGlobalStore1.01.0
138KIP-231: Improve the Required ACL of ListGroups API2.1.0
139KIP-229: DeleteGroups API1.1.0
140KIP-227 - Introduce Incremental FetchRequests to Increase Partition Scalability1.1.0
141KIP-226 - Dynamic Broker Configuration1.1.0
142KIP-154 Add Kafka Connect configuration properties for creating internal topics-225 - Use tags for consumer “records.lag” metrics1.10.11.0.0
143KIP-153: Include only client traffic in BytesOutPerSec metric224: Add configuration parameter `retries` to Streams API1.10.11.0.0
144KIP-152 - Improve diagnostics for SASL authentication failures-223 - Add per-topic min lead and per-partition lead metrics to KafkaConsumer21.0.0
145KIP-151 Expose Connector type in REST API222 - Add Consumer Group operations to Admin API20.11.0.0
146KIP-150 - Kafka-Streams Cogroup221: Enhance DSL with Connecting Topic Creation and Repartition Hint2.34.0 (WIP)
147KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner220: Add AdminClient into Kafka Streams' ClientSupplier1.1.0 (partially implemented)(WIP for 2.3.0)
148KIP-146 - Classloading Isolation in Connect219 - Improve quota communication20.11.0.0
149KIP-145 - Expose Record Headers in Kafka Connect218: Make KafkaFuture.Function java 8 lambda compatible1.1.0
150KIP-144: Exponential backoff for broker reconnect attempts215: Add topic regex support for Connect sinks1.1.00.11.0.0
151KIP-143: Controller Health Metrics214: Add zookeeper.max.in.flight.requests config to the broker1.1.00.11.0.0
152KIP-213 Support non-key joining in KTable2.4.0 (WIP)
153KIP-138: Change punctuate semantics212: Enforce set of legal characters for connector names1.01.0
154KIP-137: Enhance TopicCommand --describe to show topics marked for deletion211: Revise Expiration Semantics of Consumer Group Offsets2.10.11.0.0
155KIP-136: Add Listener name to SelectorMetrics tags210 - Provide for custom error handling when Kafka Streams fails to produce1.1.00.11.0.0
156KIP-134: Delay initial consumer group rebalance208: Add SSL support to Kafka Connect REST interface1.10.11.0.0
157KIP-133: Describe and Alter Configs Admin APIs207: Offsets returned by ListOffsetsResponse should be monotonically increasing even during a partition leader change2.20.11.0.0
158KIP-130: Expose states of active tasks to KafkaStreams public API206: Add support for UUID serialization and deserialization2.11.0.0
159KIP-129: Streams Exactly-Once Semantics205: Add all() and range() API to ReadOnlyWindowStore1.10.11.0.0
160KIP-128: Add ByteArrayConverter for Kafka Connect204 : Adding records deletion operation to the new Admin Client API1.1.00.11.0.0
161KIP-126 - Allow KafkaProducer to split and resend oversized batches.203: Add toLowerCase support to sasl.kerberos.principal.to.local rule 1.10.11.0.0
162KIP-124 - Request rate quotas202: Move merge() from StreamsBuilder to KStream10.11.0.0
163KIP-123: Allow per stream/table timestamp extractor198: Remove ZK dependency from Streams Reset Tool10.11.0.0
164KIP-122: Add Reset Consumer Group Offsets tooling197 Connect REST API should include the connector type when describing a connector1.0.11.0.0
165KIP-121196: Add KStream peek methodmetrics to Kafka Connect framework1.0.11.0.0
166KIP-120: Cleanup Kafka Streams builder API195: AdminClient.createPartitions1.0.0
167KIP-192 : Provide cleaner semantics when idempotence is enabled1.119: Drop Support for Scala 2.10 in Kafka 0.110.11.0.0
168KIP-118: Drop Support for Java 7191: KafkaConsumer.subscribe() overload that takes just Pattern12.0.0
169KIP-117: Add a public AdminClient API for Kafka admin operations190: Handle client-ids consistently between clients and brokers10.11.0.0
170KIP-189: Improve principal builder interface and add support for SASL1115: Enforce offsets.topic.replication.factor upon __consumer_offsets auto topic creation0.11.0.0
171KIP-114: KTable state stores and improved semantics188 - Add new metrics to support health checks1.0.11.0.0
172KIP-113: Support replicas movement between log directories187 - Add cumulative count metric for all Kafka rate metrics1.10.0
173KIP-112: Handle disk failure for JBOD186: Increase offsets retention default to 7 days21.0.0
174KIP-183 - Change PreferredReplicaLeaderElectionCommand to use AdminClient2.2.0
175KIP-109: Old Consumer Deprecation182: Reduce Streams DSL overloads and allow easier use of custom storage engines1.0.11.0.0
176KIP-108: Create Topic Policy180: Add a broker metric specifying the number of consumer group rebalances in progress1.10.10.2.0
177KIP-107: Add deleteRecordsBefore() API in AdminClient177: Consumer perf tool should count rebalance time10.11.0.0
178KIP-176: Remove deprecated new-consumer option for tools2.0.0
179KIP-105: Addition of Record Level for Sensors175: Additional '--describe' views for ConsumerGroupCommand1.1.0.10.2.0
180KIP-104: Granular Sensors for Streams -174 - Deprecate and remove internal converter configs in WorkerConfig2.0.0.10.2.0
181KIP-103: Separation of Internal and External traffic173: Add prefix to StreamsConfig to enable setting default internal topic configs1.0.0.10.2.0
182KIP-102 - Add close with timeout for consumers171 - Extend Consumer Group Reset Offset for Stream Application1.10.10.2.0
183KIP-101 - Alter Replication Protocol to use Leader Epoch rather than High Watermark for Truncation168: Add GlobalTopicCount and GlobalPartitionCount metric per cluster1.0.0
184KIP-167: Add interface for the state store restoration process10.11.0.0
184185

KIP

-100 - Relax Type constraints in Kafka Streams API
0.10.2.0
185KIP-99: Add Global Tables to Kafka Streams0.10.2.0

-164 - Add UnderMinIsrPartitionCount and per-partition UnderMinIsr metrics

1.0.0
186

KIP-163: Lower the Minimum Required ACL Permission of OffsetFetch

1.0.0
187KIP-162: Enable topic deletion by default1186KIP-98 - Exactly Once Delivery and Transactional Messaging0.11.0.0
187188KIP-97: Improved Kafka Client RPC Compatibility Policy0.10.2.0
188KIP-96 - Add per partition metrics for in-sync and assigned replica count0.10.2.0
189KIP-94 Session Windows0.10.2.0
190KIP-93: Improve invalid timestamp handling in Kafka Streams0.10.2.0
191KIP-92 - Add per partition lag metrics to KafkaConsumer0.10.2.0
192KIP-91 Provide Intuitive User Timeouts in The Producer2.1.0
161: streams deserialization exception handlers1.0.0
189KIP-160: Augment KStream.print(), KStream.writeAsText() to allow users pass in extra parameters in the printed string1.0.0
190KIP-157 - Add consumer config options to streams reset tool1.0.0
191KIP-156 Add option "dry run" to Streams application reset tool0.11.0.0
192KIP-155 - Add range scan for windowed state stores0.11.0.0
193KIP-154 Add Kafka Connect configuration properties for creating internal topics0.11.0.193KIP-90 - Remove zkClient dependency from Streams0.10.2.0
194KIP-89: Allow sink connectors to decouple flush and offset commit153: Include only client traffic in BytesOutPerSec metric0.1011.20.0
195KIP-88: OffsetFetch Protocol Update-152 - Improve diagnostics for SASL authentication failures1.0.0.10.2.0
196KIP-86: Configurable SASL callback handlers151 Expose Connector type in REST API0.112.0.0
197KIP-85: Dynamic JAAS configuration for Kafka clients-150 - Kafka-Streams Cogroup2.4.0 (WIP)0.10.2.0
198KIP-84: Support SASL SCRAM mechanisms149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner1.1.0 (partially implemented)
(WIP for 2.4.0)0.10.2.0
199KIP-82 - Add Record Headers146 - Classloading Isolation in Connect

0.11.0.0

200KIP-81: Bound Fetch memory usage in the consumer-145 - Expose Record Headers in Kafka Connect1.1.02.2.0 (WIP)
201KIP-79 - ListOffsetRequest/ListOffsetResponse v1 and add timestamp search methods to the new consumer144: Exponential backoff for broker reconnect attempts0.1011.10.0
202KIP-78: Cluster Id143: Controller Health Metrics0.1011.10.0
203KIP-77: Improve Kafka Streams Join Semantics140: Add administrative RPCs for adding, deleting, and listing ACLs0.1011.20.0
204KIP-75 - Add per-connector Converters138: Change punctuate semantics1.00.10.1.0
205KIP-74: Add Fetch Response Size Limit in Bytes137: Enhance TopicCommand --describe to show topics marked for deletion0.1011.10.0
206KIP-73: Replication Quotas136: Add Listener name to SelectorMetrics tags0.1011.10.0
207KIP-72: Allow putting a bound on memory consumed by Incoming request 134: Delay initial consumer group rebalance0.111.0.0
208KIP-71133: Enable log compaction and deletion to co-existDescribe and Alter Configs Admin APIs0.1011.10.0
209KIP-130: Expose states of active tasks to KafkaStreams public API1.0.0
210KIP-67129: Queryable state for Kafka StreamsStreams Exactly-Once Semantics0.1011.10.0
211KIP-66: Single Message Transforms 128: Add ByteArrayConverter for Kafka Connect0.10.2.0 / 0.11.0.0
212KIP-65: Expose timestamps to Connect-126 - Allow KafkaProducer to split and resend oversized batches.0.1011.10.0
213KIP-63: Unify store and downstream caching in streams124 - Request rate quotas0.1011.10.0
214KIP-62123: Allow consumer to send heartbeats from a background threadper stream/table timestamp extractor0.1011.10.0
215KIP-60 - Make Java client classloading more flexible122: Add Reset Consumer Group Offsets tooling0.1011.10.0
216KIP-58 - Make Log Compaction Point Configurable121: Add KStream peek method

0.

10

11.

1

0.0

217KIP-57 - Interoperable LZ4 Framing120: Cleanup Kafka Streams builder API1.0.10.0.0
218KIP-56: Allow cross origin HTTP requests on all HTTP methods119: Drop Support for Scala 2.10 in Kafka 0.110.1011.0.0
219KIP-55118: Secure Quotas for Authenticated UsersDrop Support for Java 72.0.0.10.1.0
220KIP-54: Sticky Partition Assignment Strategy117: Add a public AdminClient API for Kafka admin operations0.11.0.0
221KIP-52: Connector Control APIs115: Enforce offsets.topic.replication.factor upon __consumer_offsets auto topic creation0.11.0.00.10.0.0
222KIP-51 - List Connectors REST API114: KTable state stores and improved semantics0.1011.0.0
223KIP-113: Support replicas movement between log directories1.50 - Move Authorizer to o.a.k.common package0.10.1.0
224KIP-48 Delegation token support for Kafka112: Handle disk failure for JBOD1.10.0
225KIP-110: Add Codec for ZStandard Compression2.1.0
226KIP-43: Kafka SASL enhancements109: Old Consumer Deprecation0.1011.0.0
227KIP-42: Add Producer and Consumer Interceptors108: Create Topic Policy0.10.02.0
228KIP-41: Consumer Max Records107: Add deleteRecordsBefore() API in AdminClient0.1011.0.0
229

KIP-

40: ListGroups and DescribeGroup

106 - Change Default unclean.leader.election.enabled from True to False

0.911.0.0
230KIP-38: ZooKeeper Authentication105: Addition of Record Level for Sensors0.910.02.0
231KIP-36 - Rack aware replica assignment104: Granular Sensors for Streams 0.10.02.0
232KIP-35 - Retrieving protocol version103: Separation of Internal and External traffic0.10.02.0
233KIP-33 102 - Add a time based log indexclose with timeout for consumers0.10.12.0
234KIP-32 - Add timestamps to Kafka message101 - Alter Replication Protocol to use Leader Epoch rather than High Watermark for Truncation0.110.10.0.0
235KIP-31 - Move to relative offsets in compressed message sets100 - Relax Type constraints in Kafka Streams API0.10.02.0
236KIP-28 - Add a processor client99: Add Global Tables to Kafka Streams0.10.02.0
237KIP-26 - Add Kafka Connect framework for data import/export98 - Exactly Once Delivery and Transactional Messaging0.911.0.0
238KIP-25 - System test improvements97: Improved Kafka Client RPC Compatibility Policy0.910.02.0
239KIP-22 - Expose a Partitioner interface in the new producer96 - Add per partition metrics for in-sync and assigned replica count0.910.02.0
240KIP-21 - Dynamic Configuration94 Session Windows0.910.02.0 (WIP)
241KIP-20 Enable log preallocate to improve consume performance under windows and some old Linux file system93: Improve invalid timestamp handling in Kafka Streams0.910.02.0
242KIP-19 92 - Add a request timeout to NetworkClientper partition lag metrics to KafkaConsumer0.910.02.0
243KIP-16 - Automated Replica Lag Tuning91 Provide Intuitive User Timeouts in The Producer2.10.9.0.0
244KIP-15 - Add a close method with a timeout in the producer90 - Remove zkClient dependency from Streams0.910.02.0
245KIP-13 - Quota Design89: Allow sink connectors to decouple flush and offset commit0.910.02.0
246KIP-12 - Kafka Sasl/Kerberos and SSL implementation88: OffsetFetch Protocol Update0.10.20.9.0.0
247KIP-11 - Kafka Authorizer design86: Configurable SASL callback handlers20.9.0.0
248KIP-8 - Add a flush method to the producer API85: Dynamic JAAS configuration for Kafka clients0.910.02.0
249KIP-4 - Metadata Protocol Changes84: Support SASL SCRAM mechanisms0.10.02.0
250KIP-4 - Command line and centralized administrative operations82 - Add Record Headers0.911.0.0, 0.10.0.0, 0.10.1.0
251KIP-3 - Mirror Maker Enhancement81: Bound Fetch memory usage in the consumer2.4.0 (WIP)0.9.0.0
252KIP-2 - Refactor brokers to allow listening on multiple ports and IPs79 - ListOffsetRequest/ListOffsetResponse v1 and add timestamp search methods to the new consumer0.910.01.0
253KIP-1 - Remove support of request.required.acks78: Cluster Id0.910.01.0

...

KIP59: Proposal for a kafka broker command
254KIPComment-Sent emails to Dev discussion group. Work tracked under KAFKA-3663.
KIP-125: ZookeeperConsumerConnector to KafkaConsumer Migration and Rollback 
KIP-131 - Add access to OffsetStorageReader from SourceConnector 
KIP-135 : Send of null key to a compacted topic should throw non-retriable error back to user 
KIP 141 - ProducerRecord & SourceRecord: Add timestamp constructors 
KIP-142: Add ListTopicsRequest to efficiently list all the topics in a cluster 
KIP-148: Add a connect timeout for client 
KIP-158: Kafka Connect should allow source connectors to set topic-specific settings for new topics 
KIP-159: Introducing Rich functions to Streams 
KIP-166 - Add a tool to make amounts of replicas and leaders on brokers balanced 
KIP-169 - Lag-Aware Partition Assignment Strategy 
KIP-178: Size-based log directory selection strategy

 

KIP-185: Make exactly once in order delivery the default producer setting

 
KIP-193: Add SchemaBuilder.from(Schema) 
KIP-199: Add Kafka Connect offset tool 
KIP-201: Rationalising Policy interfaces 
KIP-209: Connection String Support 
KIP-213 Support non-key joining in KTable 
KIP-216: IQ should throw different exceptions for different errors 
KIP-217: Expose a timeout to allow an expired ZK session to be re-created 
KIP-221: Enhance KStream with Connecting Topic Creation and Repartition HintDiscussion
KIP-228 Negative record timestamp supportVoting in progress
KIP-234: add support for getting topic defaults from AdminClient 
KIP-236: Interruptible Partition Reassignment Discussion
KIP-240: AdminClient.listReassignments() AdminClient.describeReassignments() 
KIP-242: Mask password in Kafka Connect Rest API response 
KIP-250 Add Support for Quorum-based Producer Acknowledgment 
KIP-252 - Extend ACLs to allow filtering based on ip ranges and subnets 
KIP-253: Support in-order message delivery with partition expansion Discussion
KIP-254: JsonConverter Exception Handeling 
KIP-259: Improve Streams DSL Timestamp Propagation Semantics 
KIP-260: add primary join operation for Stream-Stream join (WIP)Draft
KIP-264: Add a consumer metric to record raw fetch sizeVoting in progress
KIP-271: Add NetworkClient redirectorDiscussion
KIP-273: Kafka to support using ETCD beside ZookeeperDiscussion
KIP-275 - Indicate "isClosing" in the SinkTaskContextVoting in progress
KIP-280: Enhanced log compactionDiscussion

KIP-291: Separating controller connections and requests from the data plane

Accepted
KIP-293 Add new metrics for consumer/replication fetch requestsVoting in progress
KIP-296: Add connector level configurability for producer/consumer client configs Discussion
KIP-300: Add Windowed KTable API in StreamsBuilderDiscussion
KIP-301: Schema Inferencing for JsonConverterDiscussion
KIP-304: Connect runtime mode improvements for container platformsDiscussion
KIP-307: Allow to define custom processor names with KStreams DSLDiscussion
KIP-314: KTable to GlobalKTable Bi-directional JoinDiscussion
KIP-315: Stream Join Sticky AssignorDiscussion
KIP-316: Command-line overrides for ConnectDistributed worker propertiesDiscussion
KIP-317: Add transparent data encryption functionalityDiscussion
KIP-320: Allow fetchers to detect and handle log truncationDiscussion
KIP-325: Extend Consumer Group Command to Show Beginning OffsetsVoting in Progress
KIP-326: Schedulable KTable as Graph sourceDiscussion
KIP-333: Add faster mode of rebalancingDiscussion
KIP-334 - Include partitions in exceptions raised during consumer record deserialization/validation
KIP-335: Consider configurations for KafkaStreams

Discussion

KIP-345: Introduce static membership protocol to reduce consumer rebalancesAccepted
KIP-347: Enable batching in FindCoordinatorRequestDiscussion
KIP-348 Eliminate null from SourceTask#poll()KIP-350: Allow kafka-topics.sh to take brokerid as parameter to show partitions associated with it
KIP-354: Add a Maximum Log Compaction LagAccepted
KIP-356: Add KafkaConsumer fetch-error-rate and fetch-error-total metricsDiscussion
KIP-360: Improve handling of unknown producerDiscussion
KIP-362: Support Dynamic Session WindowDiscussion
KIP-363: Allow performance tools to print final results to output fileDiscussion
KIP-369: Alternative Partitioner to Support "Always Round-Robin" SelectionDiscussion
KIP-370: Remove Orphan PartitionsDiscussion
KIP-373: Allow users to create delegation tokens for other usersDiscussion
KIP-375: Kafka Clients - make Metadata#TOPIC_EXPIRY_MS configurableDiscussion
KIP-378: Enable Dependency Injection for Kafka Streams handlersDiscussion

KIP-379: Multiple Consumer Group Management

Accepted
KIP-381: Connect: Tell about records that had their offsets flushed in callback

Voting in progress (restarted 18th January 2019, due to no votes in first attempt)

KIP-383:  Pluggable interface for SSL FactoryVoting in progress
KIP-384: Add config for incompatible changes to persistent metadataDiscussion
KIP-385: Avoid throwing away prefetched dataDiscussion
KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

Discussion

KIP-388: Add observer interface to record request and responseDiscussion
KIP-389: Introduce a configurable consumer group size limitAccepted
KIP-390: Allow fine-grained configuration for compressionDiscussion, JIRA exists with pull-request
KIP-391: Allow Producing with Offsets for Cluster ReplicationDiscussion, JIRA exists with pull-request
KIP-392: Allow consumers to fetch from closest replicaDiscussion
KIP-395: Encypt-then-MAC Delegation token metadata

KIP-396: Add Commit/List Offsets Operations to AdminClient

Discussion
KIP-397: Add methods to override fetch offsets based on timestampDiscussion
KIP-398: Support reading trust store from classpathDiscussion
KIP-399: Extend ProductionExceptionHandler to cover serialization exceptionsDiscussion
KIP-400: Improve exit status in case of errors in ConsoleProducerDiscussion
KIP-401: TransformerSupplier/ProcessorSupplier EnhancementsDiscussion
KIP-403: Increase ProducerPerformance precision by using nanoTimeDraft
KIP-405: Kafka Tiered Storage

Discussion

KIP-406: GlobalStreamThread should honor custom offset policy.Discussion
KIP-407: Kafka Connect support override worker kafka api configuration with connector configuration that post by rest api

KIP-409: Allow creating under-replicated topics and partitions

Discussion

KIP-410: Add metric for request handler thread pool utilization by request type

Discussion
KIP-408: Add asynchronous processing to Kafka StreamsDiscussion

KIP-411: Make default Kafka Connect worker task client IDs distinct

Discussion
KIP-415: Incremental Cooperative Rebalancing in Kafka ConnectDiscussion
KIP-416: Notify SourceTask of ACK'd offsets, metadataDiscussion
KIP-417: Allow JmxTool to connect to a secured RMI portDiscussion, JIRA exists with pull-request
KIP-418: A method-chaining way to branch KStreamDiscussion, JIRA exists with pull-request
KIP-419: Safely notify Kafka Connect SourceTask is stoppedDiscussion
KIP-421: Support resolving externalized secrets in AbstractConfigDiscussion
KIP-422: Add support for user/client configuration in the Kafka Admin ClientDiscussion, JIRA exists with pull-request
KIP-423: Add JoinReason to Consumer Join Group Protocol

Discussion

KIP-424: Allow suppression of intermediate events based on wall clock timeDiscussion
KIP-426: Persist Broker Id to ZookeeperDiscussion
KIP-427: Add AtMinIsr topic partition categoryDiscussion
KIP-430 - Return Authorized Operations in Describe ResponsesDiscussion
KIP-431: Support of printing additional ConsumerRecord fields in DefaultMessageFormatterDiscussion, JIRA exists with pull-request

KIP-432: Additional Broker-Side Opt-In for Default, Unsecure SASL/OAUTHBEARER Implementation

Discussion
KIP-434: Add Replica Fetcher and Log Cleaner Count MetricsDiscussion

...

77: Improve Kafka Streams Join Semantics0.10.2.0
255KIP-75 - Add per-connector Converters0.10.1.0
256KIP-74: Add Fetch Response Size Limit in Bytes0.10.1.0
257KIP-73: Replication Quotas0.10.1.0
258KIP-72: Allow putting a bound on memory consumed by Incoming request 1.0.0
259KIP-71: Enable log compaction and deletion to co-exist0.10.1.0
260KIP-70: Revise Partition Assignment Semantics on New Consumer's Subscription Change0.10.1.0
261KIP-67: Queryable state for Kafka Streams0.10.1.0
262KIP-66: Single Message Transforms for Kafka Connect0.10.2.0 / 0.11.0.0
263KIP-65: Expose timestamps to Connect0.10.1.0
264KIP-63: Unify store and downstream caching in streams0.10.1.0
265KIP-62: Allow consumer to send heartbeats from a background thread0.10.1.0
266KIP-60 - Make Java client classloading more flexible0.10.1.0
267KIP-58 - Make Log Compaction Point Configurable0.10.1.0
268KIP-57 - Interoperable LZ4 Framing0.10.0.0
269KIP-56: Allow cross origin HTTP requests on all HTTP methods0.10.0.0
270KIP-55: Secure Quotas for Authenticated Users0.10.1.0
271KIP-54: Sticky Partition Assignment Strategy0.11.0.0
272KIP-52: Connector Control APIs0.10.0.0
273KIP-51 - List Connectors REST API0.10.0.0
274KIP-48 Delegation token support for Kafka1.1.0
275KIP-45 - Standardize all client sequence interaction on j.u.Collection.0.10.0.0
276KIP-43: Kafka SASL enhancements0.10.0.0
277KIP-42: Add Producer and Consumer Interceptors0.10.0.0
278KIP-41: Consumer Max Records0.10.0.0
279KIP-40: ListGroups and DescribeGroup0.9.0.0
280KIP-38: ZooKeeper Authentication0.9.0.0
281KIP-36 - Rack aware replica assignment0.10.0.0
282KIP-35 - Retrieving protocol version0.10.0.0
283KIP-33 - Add a time based log index0.10.1.0
284KIP-32 - Add timestamps to Kafka message0.10.0.0
285KIP-31 - Move to relative offsets in compressed message sets0.10.0.0
286KIP-28 - Add a processor client0.10.0.0
287KIP-26 - Add Kafka Connect framework for data import/export0.9.0.0
288KIP-25 - System test improvements0.9.0.0
289KIP-22 - Expose a Partitioner interface in the new producer0.9.0.0
290KIP-21 - Dynamic Configuration0.9.0.0 (WIP)
291KIP-20 Enable log preallocate to improve consume performance under windows and some old Linux file system0.9.0.0
292KIP-19 - Add a request timeout to NetworkClient0.9.0.0
293KIP-16 - Automated Replica Lag Tuning0.9.0.0
294KIP-15 - Add a close method with a timeout in the producer0.9.0.0
295KIP-13 - Quota Design0.9.0.0
296KIP-12 - Kafka Sasl/Kerberos and SSL implementation0.9.0.0
297KIP-11 - Kafka Authorizer design0.9.0.0
298KIP-8 - Add a flush method to the producer API0.9.0.0
299KIP-4 - Metadata Protocol Changes0.10.0.0
300KIP-4 - Command line and centralized administrative operations0.9.0.0, 0.10.0.0, 0.10.1.0
301KIP-3 - Mirror Maker Enhancement0.9.0.0
302KIP-2 - Refactor brokers to allow listening on multiple ports and IPs0.9.0.0
303KIP-1 - Remove support of request.required.acks0.9.0.0


KIPs under discussion

Please insert new rows in sorted order (ascending by KIP number). Please move the adopted/accepted KIPs to the "Adopted KIPs" table above.

KIPComment
KIP-59: Proposal for a kafka broker commandSent emails to Dev discussion group. Work tracked under KAFKA-3663.
KIP-125: ZookeeperConsumerConnector to KafkaConsumer Migration and Rollback
KIP-131 - Add access to OffsetStorageReader from SourceConnector
KIP-135 : Send of null key to a compacted topic should throw non-retriable error back to user
KIP 141 - ProducerRecord & SourceRecord: Add timestamp constructors
KIP-142: Add ListTopicsRequest to efficiently list all the topics in a cluster
KIP-148: Add a connect timeout for client
KIP-158: Kafka Connect should allow source connectors to set topic-specific settings for new topics
KIP-159: Introducing Rich functions to Streams
KIP-166 - Add a tool to make amounts of replicas and leaders on brokers balanced
KIP-169 - Lag-Aware Partition Assignment Strategy
KIP-178: Size-based log directory selection strategy


KIP-185: Make exactly once in order delivery the default producer setting


KIP-193: Add SchemaBuilder.from(Schema)
KIP-199: Add Kafka Connect offset tool
KIP-201: Rationalising Policy interfaces
KIP-209: Connection String Support
KIP-216: IQ should throw different exceptions for different errors
KIP-217: Expose a timeout to allow an expired ZK session to be re-created
KIP-228 Negative record timestamp supportVoting in progress
KIP-234: add support for getting topic defaults from AdminClient
KIP-236: Interruptible Partition Reassignment Discussion
KIP-240: AdminClient.listReassignments() AdminClient.describeReassignments()
KIP-242: Mask password in Kafka Connect Rest API response
KIP-250 Add Support for Quorum-based Producer Acknowledgment
KIP-252 - Extend ACLs to allow filtering based on ip ranges and subnets
KIP-253: Support in-order message delivery with partition expansion Discussion
KIP-254: JsonConverter Exception Handeling
KIP-259: Improve Streams DSL Timestamp Propagation Semantics
KIP-260: add primary join operation for Stream-Stream join (WIP)Draft
KIP-264: Add a consumer metric to record raw fetch sizeVoting in progress
KIP-271: Add NetworkClient redirectorDiscussion
KIP-273: Kafka to support using ETCD beside ZookeeperDiscussion
KIP-275 - Indicate "isClosing" in the SinkTaskContextVoting in progress
KIP-280: Enhanced log compactionDiscussion
KIP-293 Add new metrics for consumer/replication fetch requestsVoting in progress
KIP-296: Add connector level configurability for producer/consumer client configs Discussion
KIP-301: Schema Inferencing for JsonConverterDiscussion
KIP-304: Connect runtime mode improvements for container platformsDiscussion
KIP-309: Add toUpperCase support to sasl.kerberos.principal.to.local ruleDiscussion
KIP-314: KTable to GlobalKTable Bi-directional JoinDiscussion
KIP-315: Stream Join Sticky AssignorDiscussion
KIP-316: Command-line overrides for ConnectDistributed worker propertiesDiscussion
KIP-317: Add end-to-end data encryption functionality to Apache KafkaDiscussion
KIP-325: Extend Consumer Group Command to Show Beginning OffsetsVoting in Progress345
KIP-326: Schedulable KTable as Graph sourceDiscussion
KIP-333: Add faster mode of rebalancingDiscussion
KIP-334 - Include partitions in exceptions raised during consumer record deserialization/validation
KIP-335: Consider configurations for KafkaStreams

Discussion

KIP-347: Enable batching in FindCoordinatorRequestDiscussion
KIP-348 Eliminate null from SourceTask#poll()
KIP-350: Allow kafka-topics.sh to take brokerid as parameter to show partitions associated with it
KIP-356: Add KafkaConsumer fetch-error-rate and fetch-error-total metricsDiscussion
KIP-360: Improve handling of unknown producerDiscussion
KIP-362: Support Dynamic Session WindowDiscussion
KIP-363: Allow performance tools to print final results to output fileDiscussion
KIP-370: Remove Orphan PartitionsDiscussion
KIP-373: Allow users to create delegation tokens for other usersDiscussion
KIP-375: Kafka Clients - make Metadata#TOPIC_EXPIRY_MS configurableDiscussion
KIP-378: Enable Dependency Injection for Kafka Streams handlersDiscussion
KIP-381: Connect: Tell about records that had their offsets flushed in callback

Voting in progress (restarted 18th January 2019, due to no votes in first attempt)

KIP-383:  Pluggable interface for SSL FactoryVoting in progress
KIP-384: Add config for incompatible changes to persistent metadataDiscussion
KIP-385: Avoid throwing away prefetched dataDiscussion
KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

Discussion

KIP-388: Add observer interface to record request and responseDiscussion
KIP-390: Allow fine-grained configuration for compressionDiscussion, JIRA exists with pull-request
KIP-391: Allow Producing with Offsets for Cluster ReplicationDiscussion, JIRA exists with pull-request
KIP-392: Allow consumers to fetch from closest replicaDiscussion
KIP-395: Encypt-then-MAC Delegation token metadata
KIP-397: Add methods to override fetch offsets based on timestampDiscussion
KIP-398: Support reading trust store from classpathDiscussion
KIP-399: Extend ProductionExceptionHandler to cover serialization exceptionsDiscussion
KIP-400: Improve exit status in case of errors in ConsoleProducerDiscussion
KIP-403: Increase ProducerPerformance precision by using nanoTimeDraft
KIP-405: Kafka Tiered Storage

Discussion

KIP-406: GlobalStreamThread should honor custom offset policy.Discussion
KIP-407: Kafka Connect support override worker kafka api configuration with connector configuration that post by rest api
KIP-408: Add asynchronous processing to Kafka StreamsDiscussion

KIP-409: Allow creating under-replicated topics and partitions


KIP-410: Add metric for request handler thread pool utilization by request type

Discussion
KIP-416: Notify SourceTask of ACK'd offsets, metadataDiscussion
KIP-418: A method-chaining way to branch KStreamDiscussion, JIRA exists with pull-request
KIP-419: Safely notify Kafka Connect SourceTask is stoppedVoting in progress, JIRA exists with pull request
KIP-422: Add support for client quota configuration in the Kafka Admin ClientDiscussion, JIRA exists with pull-request
KIP-423: Add JoinReason to Consumer Join Group Protocol

Discussion

KIP-424: Allow suppression of intermediate events based on wall clock timeDiscussion
KIP-426: Persist Broker Id to ZookeeperDiscussion
KIP-431: Support of printing additional ConsumerRecord fields in DefaultMessageFormatterVoting in progress, JIRA exists with pull-request
KIP-434: Add Replica Fetcher and Log Cleaner Count MetricsDiscussion
KIP-435: Internal Partition Reassignment BatchingDiscussion
KIP-437: Custom replacement for MaskField SMTVoting in progress, JIRA exists with pull-request
KIP-438: Expose task, connector IDs in Connect APIDiscussion
KIP-439: Cleanup built-in Store interfacesDiscussion
KIP-441: Smooth Scaling Out for Kafka StreamsDiscussion
KIP-444: Augment metrics for Kafka StreamsDiscussion
KIP-447: Producer scalability for exactly once semanticsDiscussion
KIP-448: Add State Stores Unit Test Support to Kafka Streams Test UtilsDiscussion
KIP-450: Sliding Window Aggregations in the DSLDiscussion
KIP-452: Tool to view cluster statusDiscussion, JIRA exists
KIP-457: Add DISCONNECTED status to Kafka StreamsDiscussion 
KIP-459: Improve KafkaStreams#closeDiscussion
KIP-463: Auto-configure non-default Serdes passed alongside the TopologyBuilderUnder discussion
KIP-466: Add support for List<T> serialization and deserializationVoting in progress
KIP-468: Avoid decompression of record when validate record at server in the scene of inPlaceAssignment .Under discussion
KIP-470: TopologyTestDriver test input and output usability improvementsUnder discussion
KIP-473: Enable KafkaLog4JAppender to use SASL Authentication Callback HandlersUnder Discussion 
KIP-477: Add PATCH method for connector config in Connect REST APIUnder Discussion
KIP-478 - Strongly typed Processor APIUnder Discussion
KIP-481: SerDe Improvements for Connect Decimal type in JSONUnder Discussion
KIP-486: Support custom way to load KeyStore and TrustStoreUnder Discussion
KIP-487: Automatic Topic Creation on ProducerUnder Discussion
KIP-489: Kafka Consumer Record Latency MetricUnder Discussion
KIP-490: New metric to count offsets expired without being consumed by a consumer groupUnder discussion
KIP-491: Preferred Leader Deprioritized List (Temporary Blacklist)Under discussion
KIP-494: Connect REST Endpoint to get Transformations (SMTs)
KIP-495: Dynamically Adjust Log Levels in ConnectSent mail to dev discussion group. 
KIP-498: Add client-side configuration for maximum response size to protect against OOMSent emails to Dev discussion group. Work tracked under KAFKA-4090.

KIP-499 - Unify connection name flag for command line tool

Under discussion
KIP-499: Create a controller RPC for changing configurations
KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum
KIP-502: Connect SinkTask.put(...) to specify ArrayList<SinkRecord> in Signature
KIP-505: Add new public method to only update assignment metadata in consumerUnder discussion
KIP-506: Allow setting SCRAM password via Admin interfaceUnder discussion
KIP-507: Securing Internal Connect REST EndpointsUnder discussion
KIP-508: Make Suppression State QueriableUnder discussion
KIP-509: Rebalance and restart ProducersUnder discussion
KIP-510: Metrics library upgradeUnder discussion

KIP-511: Collect and Expose Client's Name and Version in the Brokers


KIP-512: Adding headers to RecordMetaDataSent mail to dev discussion group
KIP-513: Distinguish between Key and Value serdes in scala wrapper library for kafka streams

KIP-514:Add a bounded flush() API to Kafka Producer


KIP-515: Reorganize checkpoint system in log cleaner to per partition 
KIP-516: Topic Identifiers & Topic Deletion State ImprovementsUnder discussion
KIP-517: Add consumer metric indicating time between poll callsUnder discussion
KIP-518: Allow listing consumer groups per stateUnder discussion
KIP-519: Make SSL context/engine configuration extensibleUnder discussion
KIP-521: Enable redirection of Connect's log4j messages to a file by defaultUnder discussion

KIP-522: Update BrokerApiVersionsCommand to use AdminClient

Draft
KIP-523: Add KStream#toTable to the Streams DSLUnder discussion
KIP-525 - Return topic metadata and configs in CreateTopics responseUnder discussion


Dormant/inactive KIPs

Please insert new rows in sorted order (ascending by KIP number).

Discarded KIPs

Please insert new rows in sorted order (ascending by KIP number).

KIPComment
KIP-5 - Broker Configuration ManagementSuperseded by KIP-21
KIP-7 - Security - IP Filtering
KIP-9 - SSL SupportAdopted via KAFKA-1690 but not via this KIP process
KIP-18 - JBOD SupportSuperseded by KIP-112 and KIP-113 
KIP-24 - Remove ISR information from TopicMetadataRequest and add broker level metadata request
KIP-29 - Add an IsrPropagateIntervalMs configuration to KafkaConfigNo longer need after KAFKA-2722

KIP-34 Add Partitioner Change Listener to Partitioner Interface for Multiple Use Case

Per feedback it is better do this parition() method and avoid Thread Coordination etc.
KIP-80: Kafka Rest Server
KIP-83 - Allow multiple SASL authenticated Java clients in a single JVM processAfter KIP-85, no interface changes, reduced to KAFKA-4180
KIP-111: Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.Covered by KIP-189
KIP-116: Add State Store Checkpoint Interval ConfigurationCurrently not needed as checkpointing can be done on commit interval.
KIP-127: Pluggable JAAS LoginModule configuration for SSLSimilar functionality can be implemented using a custom PrincipalBuilder.
KIP-132: Augment KStream.print to allow extra parameters in the printed string

Duplicated by KIP-160

KIP-147: Add missing type parameters to StateStoreSupplier factories and KGroupedStream/Table methodsCovered by KIP-182
KIP-165: Extend Interactive Queries for return latest update timestamp per keyCovered by KIP-258
KIP-170: Enhanced TopicCreatePolicy and introduction of TopicDeletePolicySuperseded by KIP-201: Rationalising Policy interfaces
KIP 172: Add regular-expression topic support for sink connectorSuperseded by KIP-215: Add topic regex support for Connect sinks
KIP-179: Change ReassignPartitionsCommand to use AdminClient
KIP-184: Rename LogCleaner and related classes to LogCompactorA lot of configuration change will cause more trouble and doesn't seems to be worth it
KIP 230: Name Windowing JoinsCovered by KIP-372: Naming Repartition Topics for Joins and Grouping
KIP-232: Detect outdated metadata using per-partition leaderEpoch fieldMerged into KIP-320: Allow fetchers to detect and handle log truncation
KIP-246: Connect producers and consumers should inherit worker configsWithdrawn because proposal could not be made backward compatible with existing behavior
KIP-248 - Create New ConfigCommand That Uses The New AdminClientDiscarded by author as it's partly covered in KIP-339 and KIP-422
KIP-262: Metadata should include number of state stores for taskRejected since metadata is no longer required.
KIP-263: Allow broker to skip sanity check of inactive segments on broker startupWithdrawn because the solution we agreed on does not require interface change
KIP-286: producer.send() should not block on metadata updateWithdrawn because the benefit of not having to wait for metadata is probably not worth the complexity added in producer.
KIP-288: [DISCARDED] Consumer.poll() timeout semantic change and new waitForAssignment methodDiscarded in deference to KIP-266
KIP-310: Add a Kafka Source Connector to Kafka ConnectWithdrawn in favor of KIP-382.
KIP-327: Add describe all topics API to AdminClientWithdrawn in-favor of filter support in Metadata API and KIP-142
KIP-344: The auto-generated client id should be passed to MetricsReporterBug fix so KIP not needed
KIP-364: Remove implicit Materialized, Consumed and ProducedSubsumed by KIP-365
KIP-404: Add Kafka Connect configuration parameter for disabling WADL output on OPTIONS request

Discarded, reported as a bug by

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-7759

KIP-432: Additional Broker-Side Opt-In for Default, Unsecure SASL/OAUTHBEARER Implementation

A "security sanity check tool" is a more generic and appropriate solution.
KIP-451: Make TopologyTestDriver output iterable

Discarded in favor of KIP-456

KIP-456: Helper classes to make it simpler to write test logic with TopologyTestDriverDiscarded in favor of KIP-470
KIP-472: Add header to RecordContext/ProducerRecordDiscarded due to change being far more complex than necessary for a simple bug
KIP-485: Make topic optional when using through() operations in DSLMoved to KIP-221
KIP-50 - Move Authorizer to o.a.k.common packageSuperseded by KIP-504

Discarded KIPs

KIPComment
KIP-5 - Broker Configuration ManagementSuperseded by KIP-21
KIP-7 - Security - IP Filtering 
KIP-9 - SSL SupportAdopted via KAFKA-1690 but not via this KIP process
KIP-18 - JBOD SupportSuperseded by KIP-112 and KIP-113 
KIP-24 - Remove ISR information from TopicMetadataRequest and add broker level metadata request 
KIP-29 - Add an IsrPropagateIntervalMs configuration to KafkaConfigNo longer need after KAFKA-2722

KIP-34 Add Partitioner Change Listener to Partitioner Interface for Multiple Use Case

Per feedback it is better do this parition() method and avoid Thread Coordination etc.
KIP-80: Kafka Rest Server 
KIP-83 - Allow multiple SASL authenticated Java clients in a single JVM processAfter KIP-85, no interface changes, reduced to KAFKA-4180
KIP-116: Add State Store Checkpoint Interval ConfigurationCurrently not needed as checkpointing can be done on commit interval.
KIP-127: Pluggable JAAS LoginModule configuration for SSLSimilar functionality can be implemented using a custom PrincipalBuilder.
KIP-132: Augment KStream.print to allow extra parameters in the printed string

Duplicated by KIP-160

KIP-184: Rename LogCleaner and related classes to LogCompactorA lot of configuration change will cause more trouble and doesn't seems to be worth it
KIP-111: Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.Covered by KIP-189
KIP-147: Add missing type parameters to StateStoreSupplier factories and KGroupedStream/Table methodsCovered by KIP-182
KIP-165: Extend Interactive Queries for return latest update timestamp per keyCovered by KIP-258
KIP-170: Enhanced TopicCreatePolicy and introduction of TopicDeletePolicySuperseded by KIP-201: Rationalising Policy interfaces
KIP 172: Add regular-expression topic support for sink connectorSuperseded by KIP-215: Add topic regex support for Connect sinks
KIP-179: Change ReassignPartitionsCommand to use AdminClient
KIP 230: Name Windowing JoinsCovered by KIP-372: Naming Repartition Topics for Joins and Grouping
KIP-232: Detect outdated metadata using per-partition leaderEpoch fieldMerged into KIP-320: Allow fetchers to detect and handle log truncation
KIP-246: Connect producers and consumers should inherit worker configsWithdrawn because proposal could not be made backward compatible with existing behavior
KIP-262: Metadata should include number of state stores for taskRejected since metadata is no longer required.
KIP-263: Allow broker to skip sanity check of inactive segments on broker startupWithdrawn because the solution we agreed on does not require interface change
KIP-286: producer.send() should not block on metadata updateWithdrawn because the benefit of not having to wait for metadata is probably not worth the complexity added in producer.
KIP-288: [DISCARDED] Consumer.poll() timeout semantic change and new waitForAssignment methodDiscarded in deference to KIP-266
KIP-310: Add a Kafka Source Connector to Kafka ConnectWithdrawn in favor of KIP-382.
KIP-327: Add describe all topics API to AdminClientWithdrawn in-favor of filter support in Metadata API and KIP-142
KIP-344: The auto-generated client id should be passed to MetricsReporterBug fix so KIP not needed
KIP-364: Remove implicit Materialized, Consumed and ProducedSubsumed by KIP-365
KIP-404: Add Kafka Connect configuration parameter for disabling WADL output on OPTIONS request

Discarded, reported as a bug by

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-7759

KIP-248 - Create New ConfigCommand That Uses The New AdminClientDiscarded by author as it's partly covered in KIP-339 and KIP-422

KIP Discussion Recordings

...