This page describes a proposed Kafka Improvement Proposal (KIP) process for proposing a major change to Kafka.
To create your own KIP, click on
Create from template | ||||||||
---|---|---|---|---|---|---|---|---|
|
dev@kafka.apache.org
and request permission (http://kafka.apache.org/contact). Also add an entry to the table KIPs under discussion (for Streams API KIPs, please also add it to Kafka Streams sub page)....
- Click
. Take the next available KIP number and give your proposal a descriptive heading. e.g. "KIP 42: Allow Infinite Retention With Bounded Disk Usage".Create from template templateName 54329345 templateId 54329345 title KIP-NEXT: Insert Title Here buttonLabel Create KIP - Fill in the sections as described above
- Start a [DISCUSS] thread on the Apache mailing list. Please ensure that the subject of the thread is of the format [DISCUSS] KIP-{your KIP number} {your KIP heading} The and the body contains a link to your new KIP. The discussion should happen on the mailing list, not on the wiki, since the wiki comment system doesn't work well for larger discussions. In the process of the discussion you may update the proposal. You should let people know the changes you are making. When you feel you have a finalized proposal
- Once the proposal is finalized call a [VOTE] to have the proposal adopted. These proposals are more serious than code changes and more serious even than release votes. The criteria for acceptance is lazy majority. The vote should remain open for at least 72 hours.
- Please update the KIP wiki page, and the index below, to reflect the current stage of the KIP after a vote. This acts as the permanent record indicating the result of the KIP (e.g., Accepted or Rejected). Also report the result of the KIP vote to the voting thread on the mailing list so the conclusion is clear.
KIP round-up
Next KIP Number: 425526
Use this number as the identifier for your KIP and increment this value.
Adopted KIPs
Please insert new rows in sorted order (descending by KIP number).
KIP (please keep this sorted by KIP number) | Release | |||||
---|---|---|---|---|---|---|
1 | KIP- | 420: Add Single Value Fetch in Session Stores482: The Kafka Protocol should Support Optional Tagged Fields | 2. | 24.0 (WIP) | ||
2 | KIP-414: Expose Embedded ClientIds in Kafka Streams-504 - Add new Java Authorizer Interface | 2.24.0 (WIP) | ||||
3 | KIP-402: Improve fairness in SocketServer processors503: Add metric for number of topics marked for deletion | 2.24.0 (WIP) | ||||
4 | KIP-394: Require member.id for initial join group request497: Add inter-broker API to alter ISR | 2.24.0 (WIP) | ||||
5 | KIP-393: Time windowed serde to properly deserialize changelog input topic496: Administrative API to delete consumer offsets | 2.24.0 (WIP) | ||||
6 | KIP-386: Standardize on Min/Avg/Max metrics' default value488: Clean up Sum,Count,Total Metrics | 2.24.0 | ||||
7 | KIP-382: MirrorMaker 2.0484: Expose metrics for group and transaction metadata loading duration | 2.24.0 (WIP) | ||||
8 | KIP- | 380: Detect outdated control requests and bounced brokers using broker generation480: Sticky Partitioner | 2. | 24.0 | ||
9 | KIP- | 377479: | TopicCommandAdd Materialized to | use AdminClientJoin | 2. | 24.0(WIP) |
10 | KIP-376: Implement AutoClosable on appropriate classes that want to be used in a try-with-resource statement476: Add Java AdminClient Interface | 2.4.2.2.0 | ||||
11 | KIP- | 372: Naming Repartition Topics for Joins and Grouping475: New Metrics to Measure Number of Tasks on a Connector | 2. | 14.0 (WIP) | ||
12 | KIP-371: Add a configuration to build custom SSL principal name474: To deprecate WindowStore#put(key, value) | 2.24.0 (WIP) | ||||
13 | KIP 368: Allow SASL Connections to Periodically Re-Authenticate-471: Expose RocksDB Metrics in Kafka Streams | 2. 24.0 (WIP) | ||||
14 | KIP-367 Introduce close(Duration) to Producer and AdminClient instead of close(long, TimeUnit)467: Augment ProduceResponse error messaging for specific culprit records | 2.24.0 (WIP) | ||||
15 | KIP- | 366: Make FunctionConversions deprecated465: Add Consolidated Connector Endpoint to Connect REST API | 2. | 13.0 | ||
16 | KIP-365: Materialized, Serialized, Joined, Consumed and Produced with implicit Serde464: Defaults for AdminClient#createTopic | 2.14.0 | ||||
17 | KIPKIP-361: Add Consumer Configuration to Disable Auto Topic Creation462: Use local thread id for KStreams | 2.23.0 (WIP) | ||||
18 | KIP-359: Verify leader epoch in produce requests461: Improve Replica Fetcher behavior at handling partition failure | 2.23.0 (WIP) | ||||
19 | KIP- | 358: Migrate Streams API to Duration instead of long ms times460: Admin Leader Election RPC | 2. | 14.0 | ||
20 | KIP-357: Add support to list ACLs per principal458: Connector Client Config Override Policy | 2.13.0 | ||||
21 | KIP-356: Add withCachingDisabled() to StoreBuilder455: Create an Administrative API for Replica Reassignment | 2.14.0 (WIP) | ||||
22 | KIP-353: Improve Kafka Streams Timestamp Synchronization454: Expansion of the ConnectClusterState interface | 2.13.0 | ||||
23 | KIP-351453: Add --under-min-isr option to describe topics commandclose() method to RocksDBConfigSetter | 2.23.0 (WIP) | ||||
24 | KIP- | 346: Improve LogCleaner behavior on error2.1 | ||||
25 | KIP-342 Add support for custom SASL extensions in OAuthBearer authentication | 2.1.0 | ||||
449: Add connector contexts to log messages in Connect workers | 2.3.0 | |||||
25 | KIP-446: Add changelog topic configuration to KTable suppress | 2.4 | 26 | KIP-341: Update Sticky Assignor's User Data Protocol | 2.2.0 (WIP) | |
2726 | KIP-340445: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property fileIn-memory Session Store | 2.3.0 | ||||
27 | 2.32.1.0 | |||||
28 | KIP-339: Create a new IncrementalAlterConfigs API442: Return to default max poll interval in Streams | 2.23.0 (WIP) | ||||
29 | KIP-338 Support to exclude the internal topics in kafka-topics.sh command440: Extend Connect Converter to support headers | 2.14.0 (WIP) | ||||
30 | KIP-336: Consolidate ExtendedSerializer/Serializer and ExtendedDeserializer/Deserializer436: Add a metric indicating start time | 2.13.0 | ||||
31 | KIP- | 332: Update AclCommand to use AdminClient API430 - Return Authorized Operations in Describe Responses | 2. | 13.0 | ||
32 | KIP-331 Add default implementation to close() and configure() for Serializer, Deserializer and Serde429: Kafka Consumer Incremental Rebalance Protocol | 2.24.0 (WIP) | ||||
33 | KIP- | 330428: Add | retentionPeriodin | SessionBytesStoreSupplier-memory window store | 2. | 13.0 |
34 | KIP- | 328: Ability to suppress updates for KTables427: Add AtMinIsr topic partition category (new metric & TopicCommand option) | 2. | 23.0 | (WIP)||
35 | KIP-324425: Add method to get metrics() in AdminClientsome Log4J Kafka Appender Properties for Producing to Secured Brokers | 2.13.0 | ||||
36 | KIP- | 322: Return new error code for DeleteTopics API when topic deletion disabled.421: Support resolving externalized secrets in AbstractConfig | 2.3 | 2.1.0 | ||
37 | KIP- | 321: Update TopologyDescription to better represent Source and Sink Nodes420: Add Single Value Fetch in Session Stores | 2. | 12.0 | ||
38 | KIP- | 320417: Allow | fetchers to detect and handle log truncationJmxTool to connect to a secured RMI port | 2. | 13.0 | |
39 | KIP- | 319: Replace segments with segmentInterval in WindowBytesStoreSupplier415: Incremental Cooperative Rebalancing in Kafka Connect | 2. | 13.0 | ||
40 | KIP-313: Add KStream.flatTransform and KStream.flatTransformValues 414: Expose Embedded ClientIds in Kafka Streams | 2.2.0 (partially implemented)/ 2.3 | ||||
41 | KIP-412: Extend Admin API to support dynamic application log levels | 2.4.0 (WIP) | 41||||
42 | 312 Add Overloaded StreamsBuilder Build Method to Accept java.util.Properties | 2. | 13.0 | 42|||
43 | KIP- | 308: Support dynamic update of max.connections.per.ip/max.connections.per.ip.overrides2.1.0 | ||||
43 | KIP-306: Configuration for Delaying Response to Failed Authentication | 2.1.0 | ||||
44 | KIP-305: Add Connect primitive number converters | 2.0.0 | ||||
45 | KIP-303: Add Dynamic Routing in Streams Sink | 2.0.0 | ||||
46 | KIP-302 - Enable Kafka clients to use all DNS resolved IP addresses | 2.1.0 | ||||
47 | KIP-298: Error Handling in Connect | 2.0.0 | ||||
48 | KIP-297: Externalizing Secrets for Connect Configurations | 2.0.0 | ||||
49 | KIP-295 Add Streams Configuration Allowing for Optional Topology Optimization | 2.0.0 | ||||
402: Improve fairness in SocketServer processors | 2.2.0 (partially implemented) / 2.3.0 | |||||
44 | KIP-401: TransformerSupplier/ProcessorSupplier StateStore connecting | 2.4.0 (WIP) | ||||
45 | KIP-396: Add Commit/List Offsets Operations to AdminClient | 2.4.0 (WIP) | ||||
46 | KIP-394: Require member.id for initial join group request | 2.2.0 | ||||
47 | KIP-393: Time windowed serde to properly deserialize changelog input topic | 2.2.0 | ||||
48 | KIP-389: Introduce a configurable consumer group size limit | 2.2.0 | ||||
49 | KIP-386: Standardize on Min/Avg/Max metrics' default value | 2.2.0 | ||||
50 | KIP-382: MirrorMaker 2.0 | 2.4 | 50 | KIP-294 - Enable TLS hostname verification by default | 2.0.0 | |
51 | 292: Add transformValues() method to KTable | 2. | 04.0 (WIP) | |||
52 | KIP-290: Support for Prefixed ACLs380: Detect outdated control requests and bounced brokers using broker generation | 2.02.0 | ||||
53 | KIP-289: Improve the default group id behavior in KafkaConsumer377: TopicCommand to use AdminClient | 2.2.0 | ||||
54 | KIP- | 285: Connect Rest Extension Plugin376: Implement AutoClosable on appropriate classes that want to be used in a try-with-resource statement | 2. | 02.0 | ||
55 | KIP-284: Set default retention ms for Streams repartition topics to Long.MAX_VALUE374: Add '--help' option to all available Kafka CLI commands | 2.02.0 | ||||
56 | KIP-283: Efficient Memory Usage for Down-Conversion372: Naming Repartition Topics for Joins and Grouping | 2.01.0 | ||||
57 | KIP- | 282371: Add | the listener name to the authentication contexta configuration to build custom SSL principal name | 2. | 02.0 | |
58 | KIP- | 281: ConsumerPerformance: Increase Polling Loop Timeout and Make It Reachable by the End User369: Alternative Partitioner to Support "Always Round-Robin" Selection | 2. | 04.0 | ||
59 | KIP -279: Fix log divergence between leader and follower after fast leader fail over368: Allow SASL Connections to Periodically Re-Authenticate | 2.02.0 | ||||
60 | KIP-278 - Add version option to Kafka's commands367 Introduce close(Duration) to Producer and AdminClient instead of close(long, TimeUnit) | 2.02.0 | ||||
61 | KIP- | 277 - Fine Grained ACL for CreateTopics API366: Make FunctionConversions deprecated | 2. | 01.0 | ||
62 | KIP- | 276 - Add StreamsConfig prefix for different consumers365: Materialized, Serialized, Joined, Consumed and Produced with implicit Serde | 2. | 01.0 | ||
63 | KIPKIP- | 274: Kafka Streams Skipped Records Metrics361: Add Consumer Configuration to Disable Auto Topic Creation | 2. | 03.0 | ||
64 | KIP-272: Add API version tag to broker's RequestsPerSec metric360: Improve handling of unknown producer | 2.04.0 (WIP) | ||||
65 | KIP-270 - A Scala Wrapper Library for Kafka Streams359: Verify leader epoch in produce requests | 2.04.0 (WIP) | ||||
66 | KIP- | 268358: | Simplify Kafka Streams Rebalance Metadata UpgradeMigrate Streams API to Duration instead of long ms times | 2. | 01.0 | |
67 | KIP-267357: Add Processor Unit Test Support to Kafka Streams Test Utilssupport to list ACLs per principal | 2.01.0 | ||||
68 | KIP-266: Fix consumer indefinite blocking behavior356: Add withCachingDisabled() to StoreBuilder | 2.01.0 | ||||
69 | KIP- | 265: Make Windowed Serde to public APIs354: Add a Maximum Log Compaction Lag | 2. | 03.0 | ||
70 | KIP- | 261: Add Single Value Fetch in Window Stores353: Improve Kafka Streams Timestamp Synchronization | 2. | 01.0 | ||
71 | KIP-258: Allow to Store Record Timestamps in RocksDB352: Distinguish URPs caused by reassignment | 2.24.0 (WIP) | ||||
72 | KIP- | 257 - Configurable Quota Management351: Add --under-min-isr option to describe topics command | 2. | 03.0 | ||
73 | KIP-255: OAuth Authentication via SASL/OAUTHBEARER346: Improve LogCleaner behavior on error | 2.0.01 | ||||
74 | KIP- | 251: Allow timestamp manipulation in Processor API345: Introduce static membership protocol to reduce consumer rebalances | 2.4.0 (WIP), partially available in 2.3. | 2.0.0 | ||
75 | KIP- | 249: Add Delegation Token Operations to KafkaAdminClient342 Add support for custom SASL extensions in OAuthBearer authentication | 2. | 01.0 | ||
76 | KIP- | 247: Add public test utils for Kafka Streams341: Update Sticky Assignor's User Data Protocol | 2.3 | 1.1.0 | ||
77 | KIP- | 245: Use Properties instead of StreamsConfig in KafkaStreams constructor340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file | 2. | 01.0 | ||
78 | KIP-244: Add Record Header support to Kafka Streams Processor 339: Create a new IncrementalAlterConfigs API | 2.03.00 | ||||
79 | KIP- | 243: Make ProducerConfig and ConsumerConfig constructors public338 Support to exclude the internal topics in kafka-topics.sh command | 2 | 1.1.0 | ||
80 | KIP- | 239 Add queryableStoreName() to GlobalKTable336: Consolidate ExtendedSerializer/Serializer and ExtendedDeserializer/Deserializer | 2 | 1.1.0 | ||
81 | KIP-238: Expose Kafka cluster ID in Connect REST API332: Update AclCommand to use AdminClient API | 21.1.0 | ||||
82 | KIP- | 237: More Controller Health Metrics331 Add default implementation to close() and configure() for Serializer, Deserializer and Serde | 2. | 03.0 | ||
83 | KIP-235330: Add DNS alias support for secured connectionretentionPeriod in SessionBytesStoreSupplier | 2.1.0 | ||||
84 | KIP- | 233: Simplify StreamsBuilder#addGlobalStore328: Ability to suppress updates for KTables | 2.1.0 (partially implemented) / 2.3.0 (WIP) | 1.1.0|||
85 | KIP-231: Improve the Required ACL of ListGroups API324: Add method to get metrics() in AdminClient | 2.1.0 | ||||
86 | KIP-229: DeleteGroups API322: Return new error code for DeleteTopics API when topic deletion disabled. | 21.1.0 | ||||
87 | KIP- | 227 - Introduce Incremental FetchRequests to Increase Partition Scalability321: Update TopologyDescription to better represent Source and Sink Nodes | 2 | 1.1.0 | ||
88 | KIP- | 226 - Dynamic Broker Configuration320: Allow fetchers to detect and handle log truncation | 2.1.0 (partially implemented) / 2.2 | 1.1.0 | ||
89 | KIP- | 225 - Use tags for consumer “records.lag” metrics319: Replace segments with segmentInterval in WindowBytesStoreSupplier | 2 | 1.1.0 | ||
90 | KIP- | 224313: Add | configuration parameter `retries` to Streams APIKStream.flatTransform and KStream.flatTransformValues | 2.2.0 (partially implemented) / 2.3.0 | ||
1.1.0 | 91 | KIP- | 223 - Add per-topic min lead and per-partition lead metrics to KafkaConsumer312 Add Overloaded StreamsBuilder Build Method to Accept java.util.Properties | 2. | 01.0 | |
92 | KIP-222 - Add Consumer Group operations to Admin API308: Support dynamic update of max.connections.per.ip/max.connections.per.ip.overrides | 2.01.0 | ||||
93 | KIP- | 220: Add AdminClient into Kafka Streams' ClientSupplier307: Allow to define custom processor names with KStreams DSL | 2.3.0 (partial) | 1.1.0|||
94 | KIP- | 219 - Improve quota communication306: Configuration for Delaying Response to Failed Authentication | 2. | 01.0 | ||
95 | KIP-218: Make KafkaFuture.Function java 8 lambda compatible305: Add Connect primitive number converters | 2.01.1.0 | ||||
96 | KIP- | 215303: Add | topic regex support for Connect sinksDynamic Routing in Streams Sink | 2.0 | 1.1.0 | |
97 | KIP | -214: Add zookeeper.max.in.flight.requests config to the broker-302 - Enable Kafka clients to use all DNS resolved IP addresses | 2 | 1.1.0 | ||
98 | KIP- | 212: Enforce set of legal characters for connector names300: Add Windowed KTable API in StreamsBuilder | 2.4 | 1.1.0 | ||
99 | KIP- | 211: Revise Expiration Semantics of Consumer Group Offsets298: Error Handling in Connect | 2. | 10.0 | ||
100 | KIP- | 210 - Provide for custom error handling when Kafka Streams fails to produce297: Externalizing Secrets for Connect Configurations | 2.0 | 1.1.0 | ||
101 | KIP- | 208: Add SSL support to Kafka Connect REST interface295 Add Streams Configuration Allowing for Optional Topology Optimization | 2.0 | 1.1.0 | ||
102 | KIP | -207: Offsets returned by ListOffsetsResponse should be monotonically increasing even during a partition leader change-294 - Enable TLS hostname verification by default | 2. | 20.0 | ||
103 | 206: Add support for UUID serialization and deserialization2.1.0 | Accepted | ||||
104 | KIP- | 205292: Add | alltransformValues() | and range() API to ReadOnlyWindowStore1.1method to KTable | 2.0.0 | |
105 | KIP- | 204 : Adding records deletion operation to the new Admin Client API290: Support for Prefixed ACLs | 2.0 | 1.1.0 | ||
106 | KIP- | 203: Add toLowerCase support to sasl.kerberos.principal.to.local rule289: Improve the default group id behavior in KafkaConsumer | 2.2 | 1.1.0 | ||
107 | KIP-202: Move merge() from StreamsBuilder to KStream285: Connect Rest Extension Plugin | 21.0.0 | ||||
108 | KIP- | 198: Remove ZK dependency from Streams Reset Tool284: Set default retention ms for Streams repartition topics to Long.MAX_VALUE | 2 | 1.0.0 | ||
109 | KIP- | 197 Connect REST API should include the connector type when describing a connector283: Efficient Memory Usage for Down-Conversion | 2 | 1.0.0 | ||
110 | KIP- | 196282: Add | metrics to Kafka Connect frameworkthe listener name to the authentication context | 2 | 1.0.0 | |
111 | KIP- | 195: AdminClient.createPartitions281: ConsumerPerformance: Increase Polling Loop Timeout and Make It Reachable by the End User | 2 | 1.0.0 | ||
112 | KIP-192 : Provide cleaner semantics when idempotence is enabled279: Fix log divergence between leader and follower after fast leader fail over | 21.0.0 | ||||
113 | KIP | -191: KafkaConsumer.subscribe() overload that takes just Pattern-278 - Add version option to Kafka's commands | 2 | 1.0.0 | ||
114 | KIP- | 190: Handle client-ids consistently between clients and brokers277 - Fine Grained ACL for CreateTopics API | 2 | 1.0.0 | ||
115 | KIP-189: Improve principal builder interface and add support for SASL-276 - Add StreamsConfig prefix for different consumers | 21.0.0 | ||||
116 | KIP-188 - Add new metrics to support health checks274: Kafka Streams Skipped Records Metrics | 21.0.0 | ||||
117 | KIP- | 187 - Add cumulative count metric for all Kafka rate metrics272: Add API version tag to broker's RequestsPerSec metric | 2 | 1.0.0 | ||
118 | KIP- | 186: Increase offsets retention default to 7 days270 - A Scala Wrapper Library for Kafka Streams | 2.0.0 | |||
119 | KIP-183 - Change PreferredReplicaLeaderElectionCommand to use AdminClient268: Simplify Kafka Streams Rebalance Metadata Upgrade | 2.20.0 | ||||
120 | KIP-182: Reduce Streams DSL overloads and allow easier use of custom storage engines267: Add Processor Unit Test Support to Kafka Streams Test Utils | 21.0.0 | ||||
121 | KIP- | 180: Add a broker metric specifying the number of consumer group rebalances in progress266: Fix consumer indefinite blocking behavior | 2.0 | 1.1.0 | ||
122 | KIP-177: Consumer perf tool should count rebalance time265: Make Windowed Serde to public APIs | 21.0.0 | ||||
123 | KIP- | 176: Remove deprecated new-consumer option for tools261: Add Single Value Fetch in Window Stores | 2.0.0 | |||
124 | KIP- | 175: Additional '--describe' views for ConsumerGroupCommand258: Allow to Store Record Timestamps in RocksDB | 2.3.0 (partially implemented) | 1.1.0|||
125 | KIP-174 - Deprecate and remove internal converter configs in WorkerConfig257 - Configurable Quota Management | 2.0.0 | ||||
126 | KIP- | 173: Add prefix to StreamsConfig to enable setting default internal topic configs255: OAuth Authentication via SASL/OAUTHBEARER | 2 | 1.0.0 | ||
127 | KIP-171 - Extend Consumer Group Reset Offset for Stream Application251: Allow timestamp manipulation in Processor API | 2.01.1.0 | ||||
128 | KIP-168249: Add GlobalTopicCount and GlobalPartitionCount metric per clusterDelegation Token Operations to KafkaAdminClient | 21.0.0 | ||||
129 | KIP-167247: Add interface for the state store restoration processpublic test utils for Kafka Streams | 1.01.0 | ||||
130 | KIP-164 - Add UnderMinIsrPartitionCount and per-partition UnderMinIsr metrics245: Use Properties instead of StreamsConfig in KafkaStreams constructor | 21.0.0 | ||||
131 | KIP-163: Lower the Minimum Required ACL Permission of OffsetFetch244: Add Record Header support to Kafka Streams Processor API | 21.0.0 | ||||
132 | KIP-162: Enable topic deletion by default243: Make ProducerConfig and ConsumerConfig constructors public | 1.01.0 | ||||
133 | KIP-161: streams deserialization exception handlers239 Add queryableStoreName() to GlobalKTable | 1.01.0 | ||||
134 | KIP-160: Augment KStream.print(), KStream.writeAsText() to allow users pass in extra parameters in the printed string238: Expose Kafka cluster ID in Connect REST API | 1.1.1.0.0 | ||||
135 | KIP- | 157 - Add consumer config options to streams reset tool237: More Controller Health Metrics | 2 | 1.0.0 | ||
136 | KIP-156 Add option "dry run" to Streams application reset tool235: Add DNS alias support for secured connection | 2.1.00.11.0.0 | ||||
137 | KIP- | 155 - Add range scan for windowed state stores233: Simplify StreamsBuilder#addGlobalStore | 1.1.0 | 0.11.0.0|||
138 | KIP- | 154 Add Kafka Connect configuration properties for creating internal topics231: Improve the Required ACL of ListGroups API | 2.1 | 0.11.0.0 | ||
139 | KIP-153: Include only client traffic in BytesOutPerSec metric229: DeleteGroups API | 1.1.00.11.0.0 | ||||
140 | KIP-152 - Improve diagnostics for SASL authentication failures227 - Introduce Incremental FetchRequests to Increase Partition Scalability | 1.01.0 | ||||
141 | KIP-151 Expose Connector type in REST API-226 - Dynamic Broker Configuration | 1.1.00.11.0.0 | ||||
142 | KIP-150 - Kafka-Streams Cogroup225 - Use tags for consumer “records.lag” metrics | 1.1.02.2.0 (WIP) | ||||
143 | KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner224: Add configuration parameter `retries` to Streams API | 1.1.0 (partially implemented) (WIP for 2.2.0) | ||||
144 | KIP-223 - 146 - Classloading Isolation in ConnectAdd per-topic min lead and per-partition lead metrics to KafkaConsumer | 2.0.11.0.0 | ||||
145 | KIP-145 - Expose Record Headers in Kafka Connect222 - Add Consumer Group operations to Admin API | 2.01.1.0 | ||||
146 | KIP-144: Exponential backoff for broker reconnect attempts221: Enhance DSL with Connecting Topic Creation and Repartition Hint | 2.4.0 (WIP)0.11.0.0 | ||||
147 | KIP-143: Controller Health Metrics220: Add AdminClient into Kafka Streams' ClientSupplier | 1.1.00.11.0.0 | ||||
148 | KIP-140: Add administrative RPCs for adding, deleting, and listing ACLs-219 - Improve quota communication | 20.11.0.0 | ||||
149 | KIP-138: Change punctuate semantics218: Make KafkaFuture.Function java 8 lambda compatible | 1.01.0 | ||||
150 | KIP-137: Enhance TopicCommand --describe to show topics marked for deletion215: Add topic regex support for Connect sinks | 1.10.11.0.0 | ||||
151 | KIP-136214: Add Listener name to SelectorMetrics tagszookeeper.max.in.flight.requests config to the broker | 1.1.00.11.0.0 | ||||
152 | KIP | -134: Delay initial consumer group rebalance-213 Support non-key joining in KTable | 2.4.0 (WIP) | 0.11.0.0|||
153 | KIP-133: Describe and Alter Configs Admin APIs212: Enforce set of legal characters for connector names | 1.10.11.0.0 | ||||
154 | KIP- | 130: Expose states of active tasks to KafkaStreams public API211: Revise Expiration Semantics of Consumer Group Offsets | 2. | 1.0 | ||
155 | KIP-129: Streams Exactly-Once Semantics-210 - Provide for custom error handling when Kafka Streams fails to produce | 1.1.00.11.0.0 | ||||
156 | KIP-128208: Add ByteArrayConverter for SSL support to Kafka Connect REST interface | 01.111.0.0 | ||||
157 | KIP-126 - Allow KafkaProducer to split and resend oversized batches.207: Offsets returned by ListOffsetsResponse should be monotonically increasing even during a partition leader change | 2.20.11.0.0 | ||||
158 | KIP- | 124 - Request rate quotas206: Add support for UUID serialization and deserialization | 2.1 | 0.11.0.0 | ||
159 | KIP-123: Allow per stream/table timestamp extractor205: Add all() and range() API to ReadOnlyWindowStore | 1.1.00.11.0.0 | ||||
160 | KIP-122: Add Reset Consumer Group Offsets tooling204 : Adding records deletion operation to the new Admin Client API | 1.10.11.0.0 | ||||
161 | KIP-121: Add KStream peek method203: Add toLowerCase support to sasl.kerberos.principal.to.local rule | 1.10.11.0.0 | ||||
162 | KIP-120: Cleanup Kafka Streams builder API202: Move merge() from StreamsBuilder to KStream | 1.0.0 | ||||
163 | KIP-198: Remove ZK dependency from Streams Reset Tool | 1.119: Drop Support for Scala 2.10 in Kafka 0.110.11.0.0 | ||||
164 | KIP-118: Drop Support for Java 7197 Connect REST API should include the connector type when describing a connector | 12.0.0 | ||||
165 | KIP-117196: Add a public AdminClient API for Kafka admin operationsmetrics to Kafka Connect framework | 10.11.0.0 | ||||
166 | KIP-115195: Enforce offsets.topic.replication.factor upon __consumer_offsets auto topic creationAdminClient.createPartitions | 1.0.11.0.0 | ||||
167 | KIP-114: KTable state stores and improved semantics192 : Provide cleaner semantics when idempotence is enabled | 1.0.11.0.0 | ||||
168 | KIP-113: Support replicas movement between log directories191: KafkaConsumer.subscribe() overload that takes just Pattern | 1.10.0 | ||||
169 | KIP-112190: Handle disk failure for JBODclient-ids consistently between clients and brokers | 1.0.0 | ||||
170 | KIP- | 110: Add Codec for ZStandard Compression189: Improve principal builder interface and add support for SASL | 1.0 | 2.1.0 | ||
171 | KIP-109: Old Consumer Deprecation-188 - Add new metrics to support health checks | 10.11.0.0 | ||||
172 | KIP-108: Create Topic Policy-187 - Add cumulative count metric for all Kafka rate metrics | 1.00.10.2.0 | ||||
173 | KIP-107: Add deleteRecordsBefore() API in AdminClient186: Increase offsets retention default to 7 days | 20.11.0.0 | ||||
174 | KIP-106 183 - Change Default unclean.leader.election.enabled from True to FalsePreferredReplicaLeaderElectionCommand to use AdminClient | 2.2.00.11.0.0 | ||||
175 | KIP-105: Addition of Record Level for Sensors182: Reduce Streams DSL overloads and allow easier use of custom storage engines | 1.00.10.2.0 | ||||
176 | KIP-104: Granular Sensors for Streams 180: Add a broker metric specifying the number of consumer group rebalances in progress | 1.10.10.2.0 | ||||
177 | KIP-103: Separation of Internal and External traffic177: Consumer perf tool should count rebalance time | 1.00.10.2.0 | ||||
178 | KIP- | 102 - Add close with timeout for consumers176: Remove deprecated new-consumer option for tools | 2.0 | 0.10.2.0 | ||
179 | KIP-101 - Alter Replication Protocol to use Leader Epoch rather than High Watermark for Truncation175: Additional '--describe' views for ConsumerGroupCommand | 1.1.00.11.0.0 | ||||
180 | KIP-100 - Relax Type constraints in Kafka Streams API174 - Deprecate and remove internal converter configs in WorkerConfig | 2.0.0.10.2.0 | ||||
181 | KIP-99173: Add Global Tables to Kafka Streamsprefix to StreamsConfig to enable setting default internal topic configs | 1.0.0.10.2.0 | ||||
182 | KIP-98 - Exactly Once Delivery and Transactional Messaging171 - Extend Consumer Group Reset Offset for Stream Application | 1.1.00.11.0.0 | ||||
183 | KIP-97: Improved Kafka Client RPC Compatibility Policy168: Add GlobalTopicCount and GlobalPartitionCount metric per cluster | 1.00.10.2.0 | ||||
184 | KIP-96 - Add per partition metrics for in-sync and assigned replica count167: Add interface for the state store restoration process | 1.0.0.10.2.0 | ||||
185 | -94 Session Windows -164 - Add UnderMinIsrPartitionCount and per-partition UnderMinIsr metrics | 1.00.10.2.0 | ||||
186 | 93: Improve invalid timestamp handling in Kafka Streams | 1.0. | 0.10.2.0 | |||
187 | KIP-92 - Add per partition lag metrics to KafkaConsumer162: Enable topic deletion by default | 1.00.10.2.0 | ||||
188 | KIP-91 Provide Intuitive User Timeouts in The Producer161: streams deserialization exception handlers | 1.02.1.0 | ||||
189 | KIP-90 - Remove zkClient dependency from Streams160: Augment KStream.print(), KStream.writeAsText() to allow users pass in extra parameters in the printed string | 1.00.10.2.0 | ||||
190 | KIP | -89: Allow sink connectors to decouple flush and offset commit-157 - Add consumer config options to streams reset tool | 1.0 | 0.10.2.0 | ||
191 | KIP- | 88: OffsetFetch Protocol Update156 Add option "dry run" to Streams application reset tool | 0. | 1011. | 20.0 | |
192 | KIP- | 86: Configurable SASL callback handlers155 - Add range scan for windowed state stores | 0.11 | 2.0.0 | ||
193 | KIP-85: Dynamic JAAS configuration for Kafka clients154 Add Kafka Connect configuration properties for creating internal topics | 0.1011.20.0 | ||||
194 | KIP-84: Support SASL SCRAM mechanisms153: Include only client traffic in BytesOutPerSec metric | 0.1011.20.0 | ||||
195 | KIP-82 - Add Record Headers152 - Improve diagnostics for SASL authentication failures | 10.11.0.0 | ||||
196 | KIP-81: Bound Fetch memory usage in the consumer151 Expose Connector type in REST API | 0.11.0.02.2.0 (WIP) | ||||
197 | KIP-150 - 79 - ListOffsetRequest/ListOffsetResponse v1 and add timestamp search methods to the new consumerKafka-Streams Cogroup | 2.4.0 (WIP)0.10.1.0 | ||||
198 | KIP-78: Cluster Id149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner | 1.1.0 (partially implemented) (WIP for 2.4.0)0.10.1.0 | ||||
199 | KIP-77: Improve Kafka Streams Join Semantics-146 - Classloading Isolation in Connect | 0. 1011. 20.0 | ||||
200 | KIP-75 - Add per-connector Converters145 - Expose Record Headers in Kafka Connect | 10.10.1.0 | ||||
201 | KIP-74: Add Fetch Response Size Limit in Bytes144: Exponential backoff for broker reconnect attempts | 0.1011.10.0 | ||||
202 | KIP-73: Replication Quotas143: Controller Health Metrics | 0.1011.10.0 | ||||
203 | KIP-72: Allow putting a bound on memory consumed by Incoming request 140: Add administrative RPCs for adding, deleting, and listing ACLs | 0.111.0.0 | ||||
204 | KIP-71: Enable log compaction and deletion to co-exist138: Change punctuate semantics | 1.00.10.1.0 | ||||
205 | KIP-70: Revise Partition Assignment Semantics on New Consumer's Subscription Change137: Enhance TopicCommand --describe to show topics marked for deletion | 0.1011.10.0 | ||||
206 | KIP-67: Queryable state for Kafka Streams136: Add Listener name to SelectorMetrics tags | 0.1011.10.0 | ||||
207 | KIP-66: Single Message Transforms for Kafka Connect134: Delay initial consumer group rebalance | 0.10.2.0 / 0.11.0.0 | ||||
208 | KIP-65: Expose timestamps to Connect133: Describe and Alter Configs Admin APIs | 0.1011.10.0 | ||||
209 | KIP- | 63: Unify store and downstream caching in streams130: Expose states of active tasks to KafkaStreams public API | 1.0. | 0.10.1.0 | ||
210 | KIP-62: Allow consumer to send heartbeats from a background thread129: Streams Exactly-Once Semantics | 0.1011.10.0 | ||||
211 | KIP-60 - Make Java client classloading more flexible128: Add ByteArrayConverter for Kafka Connect | 0.1011.10.0 | ||||
212 | KIP-58 - Make Log Compaction Point Configurable126 - Allow KafkaProducer to split and resend oversized batches. | 0.1011.10.0 | ||||
213 | KIP-57 - Interoperable LZ4 Framing124 - Request rate quotas | 0.1011.0.0 | ||||
214 | KIP-56123: Allow cross origin HTTP requests on all HTTP methodsper stream/table timestamp extractor | 0.1011.0.0 | ||||
215 | KIP-55: Secure Quotas for Authenticated Users122: Add Reset Consumer Group Offsets tooling | 0.1011.10.0 | ||||
216 | KIP-54: Sticky Partition Assignment Strategy121: Add KStream peek method | 0.11.0.0 | ||||
217 | KIP-52: Connector Control APIs120: Cleanup Kafka Streams builder API | 1.0.10.0.0 | ||||
218 | KIP-51 - List Connectors REST API119: Drop Support for Scala 2.10 in Kafka 0.11 | 0.1011.0.0 | ||||
219 | KIP-118: Drop Support for Java 7 | 2.0.050 - Move Authorizer to o.a.k.common package0.10.1.0 | ||||
220 | KIP-48 Delegation token support 117: Add a public AdminClient API for Kafka admin operations | 10.11.10.0 | ||||
221 | KIP-45 - Standardize all client sequence interaction on j.u.Collection.115: Enforce offsets.topic.replication.factor upon __consumer_offsets auto topic creation | 0.1011.0.0 | ||||
222 | KIP-43: Kafka SASL enhancements114: KTable state stores and improved semantics | 0.1011.0.0 | ||||
223 | KIP-42: Add Producer and Consumer Interceptors113: Support replicas movement between log directories | 1.1.00.10.0.0 | ||||
224 | KIP-41: Consumer Max Records112: Handle disk failure for JBOD | 10.10.0.0 | ||||
225 | KIP- | 40: ListGroups and DescribeGroup110: Add Codec for ZStandard Compression | 2.1.0 | 0.9.0.0|||
226 | KIP-38: ZooKeeper Authentication109: Old Consumer Deprecation | 0.911.0.0 | ||||
227 | KIP-36 - Rack aware replica assignment108: Create Topic Policy | 0.10.02.0 | ||||
228 | KIP-35 - Retrieving protocol version107: Add deleteRecordsBefore() API in AdminClient | 0.1011.0.0 | ||||
229 | 33 - Add a time based log index 106 - Change Default unclean.leader.election.enabled from True to False | 0.1011.10.0 | ||||
230 | KIP-32 - Add timestamps to Kafka message105: Addition of Record Level for Sensors | 0.10.02.0 | ||||
231 | KIP-31 - Move to relative offsets in compressed message sets104: Granular Sensors for Streams | 0.10.02.0 | ||||
232 | KIP-28 - Add a processor client103: Separation of Internal and External traffic | 0.10.02.0 | ||||
233 | KIP-26 102 - Add Kafka Connect framework for data import/exportclose with timeout for consumers | 0.910.02.0 | ||||
234 | KIP-25 - System test improvements101 - Alter Replication Protocol to use Leader Epoch rather than High Watermark for Truncation | 0.911.0.0 | ||||
235 | KIP-22 - Expose a Partitioner interface in the new producer100 - Relax Type constraints in Kafka Streams API | 0.910.02.0 | ||||
236 | KIP-21 - Dynamic Configuration99: Add Global Tables to Kafka Streams | 0.910.02.0 (WIP) | ||||
237 | KIP-20 Enable log preallocate to improve consume performance under windows and some old Linux file system-98 - Exactly Once Delivery and Transactional Messaging | 0.911.0.0 | ||||
238 | KIP-19 - Add a request timeout to NetworkClient97: Improved Kafka Client RPC Compatibility Policy | 0.910.02.0 | ||||
239 | KIP-16 - Automated Replica Lag Tuning96 - Add per partition metrics for in-sync and assigned replica count | 0.910.02.0 | ||||
240 | KIP-15 - Add a close method with a timeout in the producer94 Session Windows | 0.910.02.0 | ||||
241 | KIP-13 - Quota Design93: Improve invalid timestamp handling in Kafka Streams | 0.910.02.0 | ||||
242 | KIP-12 - Kafka Sasl/Kerberos and SSL implementation92 - Add per partition lag metrics to KafkaConsumer | 0.910.02.0 | ||||
243 | KIP-11 - Kafka Authorizer design91 Provide Intuitive User Timeouts in The Producer | 2.1.00.9.0.0 | ||||
244 | KIP-8 - Add a flush method to the producer API90 - Remove zkClient dependency from Streams | 0.910.02.0 | ||||
245 | KIP-4 - Metadata Protocol Changes89: Allow sink connectors to decouple flush and offset commit | 0.10.02.0 | ||||
246 | KIP-4 - Command line and centralized administrative operations88: OffsetFetch Protocol Update | 0.10.20.9.0.0, 0.10.0.0, 0.10.1.0 | ||||
247 | KIP-3 - Mirror Maker Enhancement86: Configurable SASL callback handlers | 2.0.9.0.0 | ||||
248 | KIP-2 - Refactor brokers to allow listening on multiple ports and IPs85: Dynamic JAAS configuration for Kafka clients | 0.910.02.0 | ||||
249 | KIP-1 - Remove support of request.required.acks84: Support SASL SCRAM mechanisms | 0.910.02.0 |
...
...
KIPs under discussion
Please insert new rows in sorted order (ascending by KIP number). Please move the adopted/accepted KIPs to the "Adopted KIPs" table above.
Dormant/inactive KIPs
Please insert new rows in sorted order (ascending by KIP number).
Discarded KIPs
Please insert new rows in sorted order (ascending by KIP number).
Discarded KIPs
KIP | Comment | |
---|---|---|
KIP-5 - Broker Configuration Management | Superseded by KIP-21 | |
KIP-7 - Security - IP Filtering | ||
KIP-9 - SSL Support | Adopted via KAFKA-1690 but not via this KIP process | |
KIP-18 - JBOD Support | Superseded by KIP-112 and KIP-113 | |
KIP-24 - Remove ISR information from TopicMetadataRequest and add broker level metadata request | ||
KIP-29 - Add an IsrPropagateIntervalMs configuration to KafkaConfig | No longer need after KAFKA-2722 | |
KIP-34 Add Partitioner Change Listener to Partitioner Interface for Multiple Use Case | Per feedback it is better do this parition() method and avoid Thread Coordination etc. | |
KIP-80: Kafka Rest Server | ||
KIP-83 - Allow multiple SASL authenticated Java clients in a single JVM process | After KIP-85, no interface changes, reduced to KAFKA-4180 | |
KIP-116: Add State Store Checkpoint Interval Configuration | Currently not needed as checkpointing can be done on commit interval. | |
KIP-127: Pluggable JAAS LoginModule configuration for SSL | Similar functionality can be implemented using a custom PrincipalBuilder. | |
KIP-132: Augment KStream.print to allow extra parameters in the printed string | Duplicated by KIP-160 | |
KIP-184: Rename LogCleaner and related classes to LogCompactor | A lot of configuration change will cause more trouble and doesn't seems to be worth it | |
KIP-111: Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker. | Covered by KIP-189 | |
KIP-147: Add missing type parameters to StateStoreSupplier factories and KGroupedStream/Table methods | Covered by KIP-182 | |
KIP-165: Extend Interactive Queries for return latest update timestamp per key | Covered by KIP-258 | |
KIP-170: Enhanced TopicCreatePolicy and introduction of TopicDeletePolicy | Superseded by KIP-201: Rationalising Policy interfaces | |
KIP 172: Add regular-expression topic support for sink connector | Superseded by KIP-215: Add topic regex support for Connect sinks | KIP-179: Change ReassignPartitionsCommand to use AdminClient |
KIP 230: Name Windowing Joins | Covered by KIP-372: Naming Repartition Topics for Joins and Grouping | |
KIP-232: Detect outdated metadata using per-partition leaderEpoch field | Merged into KIP-320: Allow fetchers to detect and handle log truncation | |
KIP-246: Connect producers and consumers should inherit worker configs | Withdrawn because proposal could not be made backward compatible with existing behavior | |
KIP-262: Metadata should include number of state stores for task | Rejected since metadata is no longer required. | |
KIP-263: Allow broker to skip sanity check of inactive segments on broker startup | Withdrawn because the solution we agreed on does not require interface change | |
KIP-286: producer.send() should not block on metadata update | Withdrawn because the benefit of not having to wait for metadata is probably not worth the complexity added in producer. | |
KIP-288: [DISCARDED] Consumer.poll() timeout semantic change and new waitForAssignment method | Discarded in deference to KIP-266 | |
KIP-310: Add a Kafka Source Connector to Kafka Connect | Withdrawn in favor of KIP-382. | |
KIP-327: Add describe all topics API to AdminClient | Withdrawn in-favor of filter support in Metadata API and KIP-142 | |
KIP-344: The auto-generated client id should be passed to MetricsReporter | Bug fix so KIP not needed | |
KIP-364: Remove implicit Materialized, Consumed and Produced | Subsumed by KIP-365 | |
KIP-404: Add Kafka Connect configuration parameter for disabling WADL output on OPTIONS request | Discarded, reported as a bug by Jira | |
server | ASF JIRA | |
serverId | 5aa69414-a9e9-3523-82ec-879b028fb15b | key | KAFKA-7759
KIP Discussion Recordings
...