This page describes a proposed Kafka Improvement Proposal (KIP) process for proposing a major change to Kafka.
To create your own KIP, click on
Create from template | ||||||||
---|---|---|---|---|---|---|---|---|
|
dev@kafka.apache.org
and request permission (http://kafka.apache.org/contact). Also add an entry to the table KIPs under discussion (for Streams API KIPs, please also add it to Kafka Streams sub page)....
- Click
. Take the next available KIP number and give your proposal a descriptive heading. e.g. "KIP 42: Allow Infinite Retention With Bounded Disk Usage".Create from template templateName 54329345 templateId 54329345 title KIP-NEXT: Insert Title Here buttonLabel Create KIP - Fill in the sections as described above
- Start a [DISCUSS] thread on the Apache mailing list. Please ensure that the subject of the thread is of the format [DISCUSS] KIP-{your KIP number} {your KIP heading} The and the body contains a link to your new KIP. The discussion should happen on the mailing list, not on the wiki, since the wiki comment system doesn't work well for larger discussions. In the process of the discussion you may update the proposal. You should let people know the changes you are making. When you feel you have a finalized proposal
- Once the proposal is finalized call a [VOTE] to have the proposal adopted. These proposals are more serious than code changes and more serious even than release votes. The criteria for acceptance is lazy majority. The vote should remain open for at least 72 hours.
- Please update the KIP wiki page, and the index below, to reflect the current stage of the KIP after a vote. This acts as the permanent record indicating the result of the KIP (e.g., Accepted or Rejected). Also report the result of the KIP vote to the voting thread on the mailing list so the conclusion is clear.
KIP round-up
Next KIP Number: 399526
Use this number as the identifier for your KIP and increment this value.
Adopted KIPs
Please insert new rows in sorted order (descending by KIP number).
KIP (please keep this sorted by KIP number) | Release | |||||
---|---|---|---|---|---|---|
1 | KIP- | 386: Standardize on Min/Avg/Max metrics' default value482: The Kafka Protocol should Support Optional Tagged Fields | 2. | 24.0 (WIP) | ||
2 | KIP-380: Detect outdated control requests and bounced brokers using broker generation-504 - Add new Java Authorizer Interface | 2.24.0 (WIP) | ||||
3 | KIP-377: TopicCommand to use AdminClient503: Add metric for number of topics marked for deletion | 2.24.0 (WIP) | ||||
4 | KIP-376: Implement AutoClosable on appropriate classes that want to be used in a try-with-resource statement497: Add inter-broker API to alter ISR | 2.24.0 (WIP) | ||||
5 | KIP-331 Add default implementation to close() and configure() for Serializer, Deserializer and Serde496: Administrative API to delete consumer offsets | 2.24.0 (WIP) | ||||
6 | KIP-372: Naming Repartition Topics for Joins and Grouping488: Clean up Sum,Count,Total Metrics | 2.14.0 | ||||
7 | KIP- | 371: Add a configuration to build custom SSL principal name484: Expose metrics for group and transaction metadata loading duration | 2. | 24.0 (WIP) | ||
8 | KIP 368: Allow SASL Connections to Periodically Re-Authenticate-480: Sticky Partitioner | 2.24.0 (WIP) | ||||
9 | KIP- | 367 Introduce close(Duration) to Producer and AdminClient instead of close(long, TimeUnit)479: Add Materialized to Join | 2. | 24.0(WIP) | ||
10 | KIP- | 366: Make FunctionConversions deprecated476: Add Java AdminClient Interface | 2. | 14.0 | ||
11 | KIP- | 365: Materialized, Serialized, Joined, Consumed and Produced with implicit Serde475: New Metrics to Measure Number of Tasks on a Connector | 2. | 14.0 (WIP) | ||
12 | KIPKIP- | 361: Add Consumer Configuration to Disable Auto Topic Creation474: To deprecate WindowStore#put(key, value) | 2. | 24.0 (WIP) | ||
13 | KIP- | 359: Verify leader epoch in produce requests471: Expose RocksDB Metrics in Kafka Streams | 2. | 24.0 (WIP) | ||
14 | KIP-358: Migrate Streams API to Duration instead of long ms times467: Augment ProduceResponse error messaging for specific culprit records | 2.24.0 (WIP) | ||||
15 | KIP- | 357465: Add | support to list ACLs per principalConsolidated Connector Endpoint to Connect REST API | 2. | 13.0 | |
16 | KIP-356: Add withCachingDisabled() to StoreBuilder464: Defaults for AdminClient#createTopic | 2.14.0 | ||||
17 | KIP-353: Improve Kafka Streams Timestamp Synchronization462: Use local thread id for KStreams | 2.13.0 | ||||
18 | KIP-346461: Improve LogCleaner behavior on errorReplica Fetcher behavior at handling partition failure | 2.13.0 | ||||
19 | KIP- | 342 Add support for custom SASL extensions in OAuthBearer authentication460: Admin Leader Election RPC | 2. | 14.0 | ||
20 | KIP- | 341: Update Sticky Assignor's User Data Protocol458: Connector Client Config Override Policy | 2. | 23.0 | (WIP)||
21 | KIP-339: Create a new IncrementalAlterConfigs API455: Create an Administrative API for Replica Reassignment | 2.24.0 (WIP) | ||||
22 | KIP-338 Support to exclude the internal topics in kafka-topics.sh command454: Expansion of the ConnectClusterState interface | 2.13.0 | ||||
23 | KIP-340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file453: Add close() method to RocksDBConfigSetter | 2.13.0 | ||||
24 | KIP- | 336: Consolidate ExtendedSerializer/Serializer and ExtendedDeserializer/Deserializer449: Add connector contexts to log messages in Connect workers | 2. | 13.0 | ||
25 | KIP- | 332: Update AclCommand to use AdminClient API446: Add changelog topic configuration to KTable suppress | 2. | 14.0 (WIP) | ||
26 | KIP-330: Add retentionPeriod in SessionBytesStoreSupplier445: In-memory Session Store | 2.13.0 | ||||
27 | 328 Ability to suppress updates for KTables | 2.23.0 (WIP) | ||||
28 | KIP-324442: Add method to get metrics() in AdminClientReturn to default max poll interval in Streams | 2.13.0 | ||||
29 | KIP- | 322: Return new error code for DeleteTopics API when topic deletion disabled.440: Extend Connect Converter to support headers | 2. | 14.0 (WIP) | ||
30 | KIP-321: Update TopologyDescription to better represent Source and Sink Nodes436: Add a metric indicating start time | 2.13.0 | ||||
31 | KIP-320: Allow fetchers to detect and handle log truncation-430 - Return Authorized Operations in Describe Responses | 2.13.0 | ||||
32 | KIP-319: Replace segments with segmentInterval in WindowBytesStoreSupplier429: Kafka Consumer Incremental Rebalance Protocol | 2.14.0 (WIP) | ||||
33 | KIP- | 313428: Add | KStream.flatTransform and KStream.flatTransformValuesin-memory window store | 2. | 23.0 | (WIP)|
34 | KIP- | 312 Add Overloaded StreamsBuilder Build Method to Accept java.util.Properties427: Add AtMinIsr topic partition category (new metric & TopicCommand option) | 2. | 13.0 | ||
35 | KIP- | 308: Support dynamic update of max.connections.per.ip/max.connections.per.ip.overrides425: Add some Log4J Kafka Appender Properties for Producing to Secured Brokers | 2.3 | 2.1.0 | ||
36 | KIP-306: Configuration for Delaying Response to Failed Authentication421: Support resolving externalized secrets in AbstractConfig | 2.13.0 | ||||
37 | KIP-305420: Add Connect primitive number convertersSingle Value Fetch in Session Stores | 2.02.0 | ||||
38 | KIP- | 303: Add Dynamic Routing in Streams Sink417: Allow JmxTool to connect to a secured RMI port | 2. | 03.0 | ||
39 | KIP- | 302 - Enable Kafka clients to use all DNS resolved IP addresses415: Incremental Cooperative Rebalancing in Kafka Connect | 2. | 13.0 | ||
40 | KIP-298: Error Handling in Connect414: Expose Embedded ClientIds in Kafka Streams | 2.02.0 | ||||
41 | KIP- | 297: Externalizing Secrets for Connect Configurations412: Extend Admin API to support dynamic application log levels | 2. | 04.0 (WIP) | ||
42 | 295 Add Streams Configuration Allowing for Optional Topology Optimization | 2. | 03.0 | |||
43 | KIP-294 - Enable TLS hostname verification by default402: Improve fairness in SocketServer processors | 2.2.0 (partially implemented) / 2.3.0 | ||||
44 | KIP-292: Add transformValues() method to KTable401: TransformerSupplier/ProcessorSupplier StateStore connecting | 2.04.0 (WIP) | ||||
45 | KIP-290: Support for Prefixed ACLs396: Add Commit/List Offsets Operations to AdminClient | 2.04.0 (WIP) | ||||
46 | KIP-289: Improve the default group id behavior in KafkaConsumer394: Require member.id for initial join group request | 2.2.0 | ||||
47 | KIP- | 285: Connect Rest Extension Plugin393: Time windowed serde to properly deserialize changelog input topic | 2. | 02.0 | ||
48 | KIP- | 284: Set default retention ms for Streams repartition topics to Long.MAX_VALUE389: Introduce a configurable consumer group size limit | 2. | 02.0 | ||
49 | KIP-283: Efficient Memory Usage for Down-Conversion386: Standardize on Min/Avg/Max metrics' default value | 2.02.0 | ||||
50 | KIP-282: Add the listener name to the authentication context382: MirrorMaker 2.0 | 2.04.0 | ||||
51 | 281: ConsumerPerformance: Increase Polling Loop Timeout and Make It Reachable by the End User | 2. | 04.0 (WIP) | |||
52 | KIP- | 279: Fix log divergence between leader and follower after fast leader fail over380: Detect outdated control requests and bounced brokers using broker generation | 2. | 02.0 | ||
53 | KIP-278 - Add version option to Kafka's commands377: TopicCommand to use AdminClient | 2.02.0 | ||||
54 | KIP-277 - Fine Grained ACL for CreateTopics API376: Implement AutoClosable on appropriate classes that want to be used in a try-with-resource statement | 2.02.0 | ||||
55 | KIP-374: Add '- | 276 - Add StreamsConfig prefix for different consumers-help' option to all available Kafka CLI commands | 2. | 02.0 | ||
56 | KIP- | 274: Kafka Streams Skipped Records Metrics372: Naming Repartition Topics for Joins and Grouping | 2. | 01.0 | ||
57 | KIP- | 272371: Add | API version tag to broker's RequestsPerSec metrica configuration to build custom SSL principal name | 2. | 02.0 | |
58 | KIP- | 270 - A Scala Wrapper Library for Kafka Streams369: Alternative Partitioner to Support "Always Round-Robin" Selection | 2. | 04.0 | ||
59 | KIP -268: Simplify Kafka Streams Rebalance Metadata Upgrade368: Allow SASL Connections to Periodically Re-Authenticate | 2.02.0 | ||||
60 | KIP- | 267: Add Processor Unit Test Support to Kafka Streams Test Utils367 Introduce close(Duration) to Producer and AdminClient instead of close(long, TimeUnit) | 2. | 02.0 | ||
61 | KIP- | 266: Fix consumer indefinite blocking behavior366: Make FunctionConversions deprecated | 2. | 01.0 | ||
62 | KIP- | 265: Make Windowed Serde to public APIs365: Materialized, Serialized, Joined, Consumed and Produced with implicit Serde | 2. | 01.0 | ||
63 | KIPKIP- | 261361: Add | Single Value Fetch in Window StoresConsumer Configuration to Disable Auto Topic Creation | 2. | 03.0 | |
64 | KIP- | 257 - Configurable Quota Management360: Improve handling of unknown producer | 2. | 04.0 (WIP) | ||
65 | KIP-255: OAuth Authentication via SASL/OAUTHBEARER359: Verify leader epoch in produce requests | 2.04.0 (WIP) | ||||
66 | KIP- | 251: Allow timestamp manipulation in Processor API358: Migrate Streams API to Duration instead of long ms times | 2. | 01.0 | ||
67 | KIP-249357: Add Delegation Token Operations to KafkaAdminClientsupport to list ACLs per principal | 2.01.0 | ||||
68 | KIP- | 247356: Add | public test utils for Kafka StreamswithCachingDisabled() to StoreBuilder | 2 | 1.1.0 | |
69 | KIP- | 245: Use Properties instead of StreamsConfig in KafkaStreams constructor354: Add a Maximum Log Compaction Lag | 2. | 03.0 | ||
70 | KIP-244: Add Record Header support to 353: Improve Kafka Streams Processor APITimestamp Synchronization | 2.01.0 | ||||
71 | KIP- | 243: Make ProducerConfig and ConsumerConfig constructors public352: Distinguish URPs caused by reassignment | 2.4.0 (WIP) | 1.1.0|||
72 | KIP- | 239 Add queryableStoreName() to GlobalKTable351: Add --under-min-isr option to describe topics command | 2.3 | 1.1.0 | ||
73 | KIP- | 238: Expose Kafka cluster ID in Connect REST API346: Improve LogCleaner behavior on error | 2.1 | 1.1.0|||
74 | KIP- | 237: More Controller Health Metrics345: Introduce static membership protocol to reduce consumer rebalances | 2.4.0 (WIP), partially available in 2.3 | 2.0.0 | ||
75 | KIP-235: 342 Add DNS alias support for secured connectioncustom SASL extensions in OAuthBearer authentication | 2.1.0 | ||||
76 | KIP- | 233: Simplify StreamsBuilder#addGlobalStore341: Update Sticky Assignor's User Data Protocol | 2.3 | 1.1.0 | ||
77 | KIP-231: Improve the Required ACL of ListGroups API340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file | 2.1.2.1.0 | ||||
78 | KIP- | 229: DeleteGroups339: Create a new IncrementalAlterConfigs API | 12. | 13. | 00 | |
79 | KIP- | 227 - Introduce Incremental FetchRequests to Increase Partition Scalability338 Support to exclude the internal topics in kafka-topics.sh command | 2 | 1.1.0 | ||
80 | KIP- | 226 - Dynamic Broker Configuration336: Consolidate ExtendedSerializer/Serializer and ExtendedDeserializer/Deserializer | 2 | 1.1.0 | ||
81 | KIP-225 - Use tags for consumer “records.lag” metrics | 1.1.0 | 82 | KIP-224: Add configuration parameter `retries` to Streams API | 332: Update AclCommand to use AdminClient API | 21.1.0 |
82 | KIP- | 223 - Add per-topic min lead and per-partition lead metrics to KafkaConsumer331 Add default implementation to close() and configure() for Serializer, Deserializer and Serde | 2.3.0 | |||
83 | KIP-330: Add retentionPeriod in SessionBytesStoreSupplier | 2.1 | 2.0.0 | |||
84 | KIP- | 222 - Add Consumer Group operations to Admin API328: Ability to suppress updates for KTables | 2.1.0 (partially implemented) / 2.3.0 (WIP) | |||
85 | KIP- | 220324: Add | AdminClient into Kafka Streams' ClientSuppliermethod to get metrics() in AdminClient | 2 | 1.1.0 | |
86 | KIP-219 - Improve quota communication322: Return new error code for DeleteTopics API when topic deletion disabled. | 2.01.0 | ||||
87 | KIP- | 218: Make KafkaFuture.Function java 8 lambda compatible321: Update TopologyDescription to better represent Source and Sink Nodes | 2 | 1.1.0 | ||
88 | KIP- | 215: Add topic regex support for Connect sinks320: Allow fetchers to detect and handle log truncation | 2.1.0 (partially implemented) / 2.2 | 1.1.0 | ||
89 | KIP- | 214: Add zookeeper.max.in.flight.requests config to the broker319: Replace segments with segmentInterval in WindowBytesStoreSupplier | 2 | 1.1.0 | ||
90 | KIP- | 212: Enforce set of legal characters for connector names1.1.0 | ||||
91 | KIP-211: Revise Expiration Semantics of Consumer Group Offsets | 2.1.0 | ||||
313: Add KStream.flatTransform and KStream.flatTransformValues | 2.2.0 (partially implemented) / 2.3.0 | |||||
91 | KIP-312 Add Overloaded StreamsBuilder Build Method to Accept java.util.Properties | 2 | 92 | KIP-210 - Provide for custom error handling when Kafka Streams fails to produce | 1.1.0 | |
9392 | KIP-208: Add SSL support to Kafka Connect REST interface308: Support dynamic update of max.connections.per.ip/max.connections.per.ip.overrides | 21.1.0 | ||||
93 | KIP- | 207: Offsets returned by ListOffsetsResponse should be monotonically increasing even during a partition leader change307: Allow to define custom processor names with KStreams DSL | 2. | 23.0 ( | WIPpartial) | |
9594 | KIP-206306: Add support for UUID serialization and deserializationConfiguration for Delaying Response to Failed Authentication | 2.1.0 | ||||
9695 | KIP-205305: Add all() and range() API to ReadOnlyWindowStoreConnect primitive number converters | 2.01.1.0 | ||||
9796 | KIP- | 204 : Adding records deletion operation to the new Admin Client API303: Add Dynamic Routing in Streams Sink | 2.0 | 1.1.0 | ||
97 | KIP- | 203: Add toLowerCase support to sasl.kerberos.principal.to.local rule302 - Enable Kafka clients to use all DNS resolved IP addresses | 2 | 1.1.0 | ||
98 | KIP- | 202: Move merge() from StreamsBuilder to KStream300: Add Windowed KTable API in StreamsBuilder | 2.4 | 1.0.0 | ||
10099 | KIP- | 198: Remove ZK dependency from Streams Reset Tool298: Error Handling in Connect | 2 | 1.0.0 | ||
101100 | KIP- | 197 Connect REST API should include the connector type when describing a connector297: Externalizing Secrets for Connect Configurations | 2 | 1.0.0 | ||
102101 | KIP- | 196: Add metrics to Kafka Connect framework295 Add Streams Configuration Allowing for Optional Topology Optimization | 2 | 1.0.0 | ||
103102 | KIP | -195: AdminClient.createPartitions-294 - Enable TLS hostname verification by default | 2 | 1.0.0 | ||
103 | Accepted | |||||
104 | KIP- | 192 : Provide cleaner semantics when idempotence is enabled292: Add transformValues() method to KTable | 2 | 1.0.0 | ||
105 | KIP- | 191: KafkaConsumer.subscribe() overload that takes just Pattern290: Support for Prefixed ACLs | 2 | 1.0.0 | ||
106 | KIP- | 190: Handle client-ids consistently between clients and brokers289: Improve the default group id behavior in KafkaConsumer | 2.2 | 1.0.0 | ||
107 | KIP-189: Improve principal builder interface and add support for SASL285: Connect Rest Extension Plugin | 21.0.0 | ||||
108 | KIP- | 188 - Add new metrics to support health checks284: Set default retention ms for Streams repartition topics to Long.MAX_VALUE | 2 | 1.0.0 | ||
109 | KIP- | 187 - Add cumulative count metric for all Kafka rate metrics283: Efficient Memory Usage for Down-Conversion | 2 | 1.0.0 | ||
110 | KIP- | 186: Increase offsets retention default to 7 days282: Add the listener name to the authentication context | 2.0.0 | |||
111 | KIP- | 183 - Change PreferredReplicaLeaderElectionCommand to use AdminClient281: ConsumerPerformance: Increase Polling Loop Timeout and Make It Reachable by the End User | 2 | 2.0.0 | ||
112 | KIP-182: Reduce Streams DSL overloads and allow easier use of custom storage engines279: Fix log divergence between leader and follower after fast leader fail over | 21.0.0 | ||||
113 | KIP-278 - | 180: Add a broker metric specifying the number of consumer group rebalances in progressAdd version option to Kafka's commands | 2.0 | 1.1.0 | ||
114 | KIP- | 177: Consumer perf tool should count rebalance time277 - Fine Grained ACL for CreateTopics API | 2 | 1.0.0 | ||
115 | KIP- | 176: Remove deprecated new-consumer option for tools276 - Add StreamsConfig prefix for different consumers | 2.0.0 | |||
116 | KIP-175: Additional '--describe' views for ConsumerGroupCommand274: Kafka Streams Skipped Records Metrics | 2.01.1.0 | ||||
117 | KIP- | 174 - Deprecate and remove internal converter configs in WorkerConfig272: Add API version tag to broker's RequestsPerSec metric | 2.0.0 | |||
118 | KIP | -173: Add prefix to StreamsConfig to enable setting default internal topic configs-270 - A Scala Wrapper Library for Kafka Streams | 2 | 1.0.0 | ||
119 | KIP-171 - Extend Consumer Group Reset Offset for Stream Application268: Simplify Kafka Streams Rebalance Metadata Upgrade | 2.01.1.0 | ||||
120 | KIP-168: Add GlobalTopicCount and GlobalPartitionCount metric per cluster267: Add Processor Unit Test Support to Kafka Streams Test Utils | 21.0.0 | ||||
121 | KIP- | 167: Add interface for the state store restoration process266: Fix consumer indefinite blocking behavior | 2 | 1.0.0 | ||
122 | KIP-164 - Add UnderMinIsrPartitionCount and per-partition UnderMinIsr metrics265: Make Windowed Serde to public APIs | 21.0.0 | ||||
123 | KIP- | 163: Lower the Minimum Required ACL Permission of OffsetFetch261: Add Single Value Fetch in Window Stores | 2 | 1.0.0 | ||
124 | KIP- | 162: Enable topic deletion by default258: Allow to Store Record Timestamps in RocksDB | 2.3.0 (partially implemented) | 1.0.0|||
125 | KIP-161: streams deserialization exception handlers257 - Configurable Quota Management | 21.0.0 | ||||
126 | KIP- | 160: Augment KStream.print(), KStream.writeAsText() to allow users pass in extra parameters in the printed string255: OAuth Authentication via SASL/OAUTHBEARER | 2 | 1.0.0 | ||
127 | KIP- | 157 - Add consumer config options to streams reset tool251: Allow timestamp manipulation in Processor API | 2 | 1.0.0 | ||
128 | KIP- | 156 Add option "dry run" to Streams application reset tool249: Add Delegation Token Operations to KafkaAdminClient | 2 | 0.11.0.0 | ||
129 | KIP- | 155 - Add range scan for windowed state stores247: Add public test utils for Kafka Streams | 1.1 | 0.11.0.0 | ||
130 | KIP-154 Add Kafka Connect configuration properties for creating internal topics245: Use Properties instead of StreamsConfig in KafkaStreams constructor | 2.0.11.0.0 | ||||
131 | KIP- | 153: Include only client traffic in BytesOutPerSec metric244: Add Record Header support to Kafka Streams Processor API | 2 | 0.11.0.0 | ||
132 | KIP-152 - Improve diagnostics for SASL authentication failures243: Make ProducerConfig and ConsumerConfig constructors public | 1.01.0 | ||||
133 | KIP-151 Expose Connector type in REST API239 Add queryableStoreName() to GlobalKTable | 1.1.00.11.0.0 | ||||
134 | KIP-150 - Kafka-Streams Cogroup238: Expose Kafka cluster ID in Connect REST API | 1.1.02.2.0 (WIP) | ||||
135 | KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner237: More Controller Health Metrics | 2.0.01.1.0 (partially implemented) (WIP for 2.2.0) | ||||
136 | KIP- | 146 - Classloading Isolation in Connect235: Add DNS alias support for secured connection | 2.1 | 0.11.0.0 | ||
137 | KIP-145 - Expose Record Headers in Kafka Connect233: Simplify StreamsBuilder#addGlobalStore | 1.1.0 | ||||
138 | KIP- | 144: Exponential backoff for broker reconnect attempts231: Improve the Required ACL of ListGroups API | 2.1.0 | 0.11.0.0|||
139 | KIP-143: Controller Health Metrics229: DeleteGroups API | 1.10.11.0.0 | ||||
140 | KIP-140: Add administrative RPCs for adding, deleting, and listing ACLs227 - Introduce Incremental FetchRequests to Increase Partition Scalability | 1.10.11.0.0 | ||||
141 | KIP-138: Change punctuate semantics226 - Dynamic Broker Configuration | 1.01.0 | ||||
142 | KIP-137: Enhance TopicCommand --describe to show topics marked for deletion225 - Use tags for consumer “records.lag” metrics | 1.1.00.11.0.0 | ||||
143 | KIP-136224: Add Listener name to SelectorMetrics tagsconfiguration parameter `retries` to Streams API | 1.10.11.0.0 | ||||
144 | KIP-134: Delay initial consumer group rebalance-223 - Add per-topic min lead and per-partition lead metrics to KafkaConsumer | 20.11.0.0 | ||||
145 | KIP-133: Describe and Alter Configs Admin APIs-222 - Add Consumer Group operations to Admin API | 2.0.11.0.0 | ||||
146 | KIP- | 130: Expose states of active tasks to KafkaStreams public API221: Enhance DSL with Connecting Topic Creation and Repartition Hint | 2.4.0 (WIP) | 1.0.0|||
147 | KIP-129220: Streams Exactly-Once SemanticsAdd AdminClient into Kafka Streams' ClientSupplier | 1.10.11.0.0 | ||||
148 | KIP-128: Add ByteArrayConverter for Kafka Connect-219 - Improve quota communication | 20.11.0.0 | ||||
149 | KIP-126 - Allow KafkaProducer to split and resend oversized batches.218: Make KafkaFuture.Function java 8 lambda compatible | 1.1.00.11.0.0 | ||||
150 | KIP-124 - Request rate quotas215: Add topic regex support for Connect sinks | 1.10.11.0.0 | ||||
151 | KIP-123: Allow per stream/table timestamp extractor214: Add zookeeper.max.in.flight.requests config to the broker | 1.10.11.0.0 | ||||
152 | KIP- | 122: Add Reset Consumer Group Offsets tooling213 Support non-key joining in KTable | 2.4.0 (WIP) | 0.11.0.0|||
153 | KIP-121: Add KStream peek method212: Enforce set of legal characters for connector names | 1.10.11.0.0 | ||||
154 | KIP-120: Cleanup Kafka Streams builder API211: Revise Expiration Semantics of Consumer Group Offsets | 2.1.0.0 | ||||
155 | KIP-210 - Provide for custom error handling when Kafka Streams fails to produce | 1.1119: Drop Support for Scala 2.10 in Kafka 0.110.11.0.0 | ||||
156 | KIP-118: Drop Support for Java 7208: Add SSL support to Kafka Connect REST interface | 1.12.0.0 | ||||
157 | KIP-117: Add a public AdminClient API for Kafka admin operations207: Offsets returned by ListOffsetsResponse should be monotonically increasing even during a partition leader change | 2.20.11.0.0 | ||||
158 | KIP- | 115: Enforce offsets.topic.replication.factor upon __consumer_offsets auto topic creation206: Add support for UUID serialization and deserialization | 2.1.0 | |||
159 | KIP-205: Add all() and range() API to ReadOnlyWindowStore | 1.1.0 | 0.11.0.0 | 159 | KIP-114: KTable state stores and improved semantics | 0.11.0.0 |
160 | KIP-113: Support replicas movement between log directories204 : Adding records deletion operation to the new Admin Client API | 1.1.0 | ||||
161 | KIP-112: Handle disk failure for JBOD203: Add toLowerCase support to sasl.kerberos.principal.to.local rule | 1.01.0 | ||||
162 | KIP- | 110: Add Codec for ZStandard Compression202: Move merge() from StreamsBuilder to KStream | 1.0 | 2.1.0 | ||
163 | KIP-109: Old Consumer Deprecation198: Remove ZK dependency from Streams Reset Tool | 10.11.0.0 | ||||
164 | KIP-108: Create Topic Policy197 Connect REST API should include the connector type when describing a connector | 1.0.0.10.2.0 | ||||
165 | KIP-107196: Add deleteRecordsBefore() API in AdminClientmetrics to Kafka Connect framework | 1.0.11.0.0 | ||||
166 | KIP-106 - Change Default unclean.leader.election.enabled from True to False195: AdminClient.createPartitions | 10.11.0.0 | ||||
167 | KIP-105: Addition of Record Level for Sensors192 : Provide cleaner semantics when idempotence is enabled | 1.00.10.2.0 | ||||
168 | KIP-104: Granular Sensors for Streams 191: KafkaConsumer.subscribe() overload that takes just Pattern | 1.0.00.10.2.0 | ||||
169 | KIP-103: Separation of Internal and External traffic190: Handle client-ids consistently between clients and brokers | 1.00.10.2.0 | ||||
170 | KIP-102 - Add close with timeout for consumers189: Improve principal builder interface and add support for SASL | 1.0.0.10.2.0 | ||||
171 | KIP-101 - Alter Replication Protocol to use Leader Epoch rather than High Watermark for Truncation188 - Add new metrics to support health checks | 10.11.0.0 | ||||
172 | KIP-100 - Relax Type constraints in Kafka Streams API187 - Add cumulative count metric for all Kafka rate metrics | 1.00.10.2.0 | ||||
173 | KIP-99: Add Global Tables to Kafka Streams186: Increase offsets retention default to 7 days | 2.0.0.10.2.0 | ||||
174 | KIP-98 - Exactly Once Delivery and Transactional Messaging183 - Change PreferredReplicaLeaderElectionCommand to use AdminClient | 2.20.11.0.0 | ||||
175 | KIP-97: Improved Kafka Client RPC Compatibility Policy182: Reduce Streams DSL overloads and allow easier use of custom storage engines | 1.00.10.2.0 | ||||
176 | KIP-96 - Add per partition metrics for in-sync and assigned replica count180: Add a broker metric specifying the number of consumer group rebalances in progress | 1.1.0.10.2.0 | ||||
177 | KIP-94 Session Windows177: Consumer perf tool should count rebalance time | 1.00.10.2.0 | ||||
178 | KIP- | 93: Improve invalid timestamp handling in Kafka Streams176: Remove deprecated new-consumer option for tools | 2.0. | 0.10.2.0 | ||
179 | KIP-92 - Add per partition lag metrics to KafkaConsumer175: Additional '--describe' views for ConsumerGroupCommand | 1.1.0.10.2.0 | ||||
180 | KIP-91 Provide Intuitive User Timeouts in The Producer174 - Deprecate and remove internal converter configs in WorkerConfig | 2.10.0 | ||||
181 | KIP-90 - Remove zkClient dependency from Streams173: Add prefix to StreamsConfig to enable setting default internal topic configs | 1.00.10.2.0 | ||||
182 | KIP-89: Allow sink connectors to decouple flush and offset commit-171 - Extend Consumer Group Reset Offset for Stream Application | 1.10.10.2.0 | ||||
183 | KIP-88: OffsetFetch Protocol Update168: Add GlobalTopicCount and GlobalPartitionCount metric per cluster | 1.0.0.10.2.0 | ||||
184 | KIP-86: Configurable SASL callback handlers167: Add interface for the state store restoration process | 12.0.0 | ||||
185 | -85: Dynamic JAAS configuration for Kafka clients -164 - Add UnderMinIsrPartitionCount and per-partition UnderMinIsr metrics | 1.00.10.2.0 | ||||
186 | 84: Support SASL SCRAM mechanisms | 1.0 | 0.10.2.0 | |||
187 | KIP-82 - Add Record Headers162: Enable topic deletion by default | 10.11.0.0 | ||||
188 | KIP-81: Bound Fetch memory usage in the consumer161: streams deserialization exception handlers | 1.0.02.2.0 (WIP) | ||||
189 | KIP-79 - ListOffsetRequest/ListOffsetResponse v1 and add timestamp search methods to the new consumer160: Augment KStream.print(), KStream.writeAsText() to allow users pass in extra parameters in the printed string | 1.0.0.10.1.0 | ||||
190 | KIP- | 78: Cluster Id157 - Add consumer config options to streams reset tool | 1.0. | 0.10.1.0 | ||
191 | KIP- | 77: Improve Kafka Streams Join Semantics156 Add option "dry run" to Streams application reset tool | 0. | 1011. | 20.0 | |
192 | KIP- | 75155 - Add | per-connector Convertersrange scan for windowed state stores | 0. | 1011. | 10.0 |
193 | KIP-74: Add Fetch Response Size Limit in Bytes154 Add Kafka Connect configuration properties for creating internal topics | 0.1011.10.0 | ||||
194 | KIP-73: Replication Quotas153: Include only client traffic in BytesOutPerSec metric | 0.1011.10.0 | ||||
195 | KIP-72: Allow putting a bound on memory consumed by Incoming request -152 - Improve diagnostics for SASL authentication failures | 1.0.0 | ||||
196 | KIP-71: Enable log compaction and deletion to co-exist151 Expose Connector type in REST API | 0.1011.10.0 | ||||
197 | KIP-70: Revise Partition Assignment Semantics on New Consumer's Subscription Change-150 - Kafka-Streams Cogroup | 2.4.0 (WIP)0.10.1.0 | ||||
198 | KIP-67: Queryable state for Kafka Streams149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner | 10.10.1.0 (partially implemented) (WIP for 2.4.0) | ||||
199 | KIP-146 - Classloading Isolation in Connect | 0. | 199 | KIP-66: Single Message Transforms for Kafka Connect | 0.10.2.0 / 0. 11.0.0 | |
200 | KIP-145 - 65: Expose timestamps to Expose Record Headers in Kafka Connect | 0.101.1.0 | ||||
201 | KIP-63: Unify store and downstream caching in streams144: Exponential backoff for broker reconnect attempts | 0.1011.10.0 | ||||
202 | KIP-62: Allow consumer to send heartbeats from a background thread143: Controller Health Metrics | 0.1011.10.0 | ||||
203 | KIP-60 - Make Java client classloading more flexible140: Add administrative RPCs for adding, deleting, and listing ACLs | 0.1011.10.0 | ||||
204 | KIP-58 - Make Log Compaction Point Configurable138: Change punctuate semantics | 1.0.0.10.1.0 | ||||
205 | KIP-57 - Interoperable LZ4 Framing137: Enhance TopicCommand --describe to show topics marked for deletion | 0.1011.0.0 | ||||
206 | KIP-56: Allow cross origin HTTP requests on all HTTP methods136: Add Listener name to SelectorMetrics tags | 0.1011.0.0 | ||||
207 | KIP-55: Secure Quotas for Authenticated Users134: Delay initial consumer group rebalance | 0.1011.10.0 | ||||
208 | KIP-54: Sticky Partition Assignment Strategy133: Describe and Alter Configs Admin APIs | 0.11.0.0 | ||||
209 | KIP- | 52: Connector Control APIs130: Expose states of active tasks to KafkaStreams public API | 1 | 0.10.0.0 | ||
210 | KIP-51 - List Connectors REST API129: Streams Exactly-Once Semantics | 0.1011.0.0 | ||||
211 | KIP-50 - Move Authorizer to o.a.k.common package128: Add ByteArrayConverter for Kafka Connect | 0.1011.10.0 | ||||
212 | KIP-48 Delegation token support for Kafka126 - Allow KafkaProducer to split and resend oversized batches. | 0.11.01.1.0 | ||||
213 | KIP-45 - Standardize all client sequence interaction on j.u.Collection.124 - Request rate quotas | 0.1011.0.0 | ||||
214 | KIP-43: Kafka SASL enhancements123: Allow per stream/table timestamp extractor | 0.1011.0.0 | ||||
215 | KIP-42122: Add Producer and Consumer InterceptorsReset Consumer Group Offsets tooling | 0.1011.0.0 | ||||
216 | KIP-41: Consumer Max Records121: Add KStream peek method | 0. 1011.0.0 | ||||
217 | KIP-40: ListGroups and DescribeGroup120: Cleanup Kafka Streams builder API | 10.9.0.0 | ||||
218 | KIP-38: ZooKeeper Authentication119: Drop Support for Scala 2.10 in Kafka 0.11 | 0.911.0.0 | ||||
219 | KIP-36 - Rack aware replica assignment118: Drop Support for Java 7 | 20.10.0.0 | ||||
220 | KIP-35 - Retrieving protocol version117: Add a public AdminClient API for Kafka admin operations | 0.1011.0.0 | ||||
221 | KIP-33 - Add a time based log index115: Enforce offsets.topic.replication.factor upon __consumer_offsets auto topic creation | 0.1011.10.0 | ||||
222 | KIP-32 - Add timestamps to Kafka message114: KTable state stores and improved semantics | 0.1011.0.0 | ||||
223 | KIP-31 - Move to relative offsets in compressed message sets113: Support replicas movement between log directories | 1.1.00.10.0.0 | ||||
224 | KIP-28 - Add a processor client112: Handle disk failure for JBOD | 10.10.0.0 | ||||
225 | KIP- | 26 - Add Kafka Connect framework for data import/export110: Add Codec for ZStandard Compression | 2.1 | 0.9.0.0 | ||
226 | KIP-25 - System test improvements109: Old Consumer Deprecation | 0.911.0.0 | ||||
227 | KIP-22 - Expose a Partitioner interface in the new producer108: Create Topic Policy | 0.910.02.0 | ||||
228 | KIP-21 - Dynamic Configuration107: Add deleteRecordsBefore() API in AdminClient | 0.911.0.0 (WIP) | ||||
229 | 20 Enable log preallocate to improve consume performance under windows and some old Linux file system 106 - Change Default unclean.leader.election.enabled from True to False | 0.911.0.0 | ||||
230 | KIP-19 - Add a request timeout to NetworkClient105: Addition of Record Level for Sensors | 0.910.02.0 | ||||
231 | KIP-16 - Automated Replica Lag Tuning104: Granular Sensors for Streams | 0.910.02.0 | ||||
232 | KIP-15 - Add a close method with a timeout in the producer103: Separation of Internal and External traffic | 0.910.02.0 | ||||
233 | KIP-13 - Quota Design102 - Add close with timeout for consumers | 0.910.02.0 | ||||
234 | KIP-12 - Kafka Sasl/Kerberos and SSL implementation101 - Alter Replication Protocol to use Leader Epoch rather than High Watermark for Truncation | 0.911.0.0 | ||||
235 | KIP-11 100 - Kafka Authorizer designRelax Type constraints in Kafka Streams API | 0.910.02.0 | ||||
236 | KIP-8 - Add a flush method to the producer API99: Add Global Tables to Kafka Streams | 0.910.02.0 | ||||
237 | KIP-4 - Metadata Protocol Changes98 - Exactly Once Delivery and Transactional Messaging | 0.1011.0.0 | ||||
238 | KIP-4 - Command line and centralized administrative operations97: Improved Kafka Client RPC Compatibility Policy | 0.9.0.0, 0.10.02.0, | ||||
239 | KIP-96 - Add per partition metrics for in-sync and assigned replica count | 0.10.12.0 | ||||
239240 | KIP-3 - Mirror Maker Enhancement94 Session Windows | 0.910.02.0 | ||||
240241 | KIP-2 - Refactor brokers to allow listening on multiple ports and IPs93: Improve invalid timestamp handling in Kafka Streams | 0.910.02.0 | ||||
241242 | KIP-1 - Remove support of request.required.acks92 - Add per partition lag metrics to KafkaConsumer | 0.910.02.0 |
...
...
KIPs under discussion
Please insert new rows in sorted order (ascending by KIP number). Please move the adopted/accepted KIPs to the "Adopted KIPs" table above.
Dormant/inactive KIPs
Please insert new rows in sorted order (ascending by KIP number).
Discarded KIPs
Please insert new rows in sorted order (ascending by KIP number).
...
Discarded KIPs
KIP | Comment |
---|---|
KIP-5 - Broker Configuration Management | Superseded by KIP-21 |
KIP-7 - Security - IP Filtering | |
KIP-9 - SSL Support | Adopted via KAFKA-1690 but not via this KIP process |
KIP-18 - JBOD Support | Superseded by KIP-112 and KIP-113 |
KIP-24 - Remove ISR information from TopicMetadataRequest and add broker level metadata request | |
KIP-29 - Add an IsrPropagateIntervalMs configuration to KafkaConfig | No longer need after KAFKA-2722 |
KIP-34 Add Partitioner Change Listener to Partitioner Interface for Multiple Use Case | Per feedback it is better do this parition() method and avoid Thread Coordination etc. |
KIP-80: Kafka Rest Server | |
KIP-83 - Allow multiple SASL authenticated Java clients in a single JVM process | After KIP-85, no interface changes, reduced to KAFKA-4180 |
KIP-116: Add State Store Checkpoint Interval Configuration | Currently not needed as checkpointing can be done on commit interval. |
KIP-127: Pluggable JAAS LoginModule configuration for SSL | Similar functionality can be implemented using a custom PrincipalBuilder. |
KIP-132: Augment KStream.print to allow extra parameters in the printed string | Duplicated by KIP-160 |
KIP-184: Rename LogCleaner and related classes to LogCompactor | A lot of configuration change will cause more trouble and doesn't seems to be worth it |
KIP-111: Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker. | Covered by KIP-189 |
KIP-147: Add missing type parameters to StateStoreSupplier factories and KGroupedStream/Table methods | Covered by KIP-182 |
KIP-170: Enhanced TopicCreatePolicy and introduction of TopicDeletePolicy | Superseded by KIP-201: Rationalising Policy interfaces |
KIP 172: Add regular-expression topic support for sink connector | Superseded by KIP-215: Add topic regex support for Connect sinks | KIP-179: Change ReassignPartitionsCommand to use AdminClient |
KIP 230: Name Windowing Joins | Covered by KIP-372: Naming Repartition Topics for Joins and Grouping |
KIP-232: Detect outdated metadata using per-partition leaderEpoch field | Merged into KIP-320: Allow fetchers to detect and handle log truncation |
KIP-246: Connect producers and consumers should inherit worker configs | Withdrawn because proposal could not be made backward compatible with existing behavior |
KIP-263: Allow broker to skip sanity check of inactive segments on broker startup | Withdrawn because the solution we agreed on does not require interface change |
KIP-286: producer.send() should not block on metadata update | Withdrawn because the benefit of not having to wait for metadata is probably not worth the complexity added in producer. |
KIP-288: [DISCARDED] Consumer.poll() timeout semantic change and new waitForAssignment method | Discarded in deference to KIP-266 |
KIP-310: Add a Kafka Source Connector to Kafka Connect | Withdrawn in favor of KIP-382. |
KIP-327: Add describe all topics API to AdminClient | Withdrawn in-favor of filter support in Metadata API and KIP-142 |
KIP-344: The auto-generated client id should be passed to MetricsReporter | Bug fix so KIP not needed |
KIP-364: Remove implicit Materialized, Consumed and Produced | Subsumed by KIP-365 |
KIP Discussion Recordings
...