This page describes a proposed Kafka Improvement Proposal (KIP) process for proposing a major change to Kafka.
To create your own KIP, click on
Create from template | ||||||||
---|---|---|---|---|---|---|---|---|
|
dev@kafka.apache.org
and request permission (http://kafka.apache.org/contact). Also add an entry to the table KIPs under discussion (for Streams API KIPs, please also add it to Kafka Streams sub page)....
- Click
. Take the next available KIP number and give your proposal a descriptive heading. e.g. "KIP 42: Allow Infinite Retention With Bounded Disk Usage".Create from template templateName 54329345 templateId 54329345 title KIP-NEXT: Insert Title Here buttonLabel Create KIP - Fill in the sections as described above
- Start a [DISCUSS] thread on the Apache mailing list. Please ensure that the subject of the thread is of the format [DISCUSS] KIP-{your KIP number} {your KIP heading} The and the body contains a link to your new KIP. The discussion should happen on the mailing list, not on the wiki, since the wiki comment system doesn't work well for larger discussions. In the process of the discussion you may update the proposal. You should let people know the changes you are making. When you feel you have a finalized proposal
- Once the proposal is finalized call a [VOTE] to have the proposal adopted. These proposals are more serious than code changes and more serious even than release votes. The criteria for acceptance is lazy majority. The vote should remain open for at least 72 hours.
- Please update the KIP wiki page, and the index below, to reflect the current stage of the KIP after a vote. This acts as the permanent record indicating the result of the KIP (e.g., Accepted or Rejected). Also report the result of the KIP vote to the voting thread on the mailing list so the conclusion is clear.
KIP round-up
Next KIP Number: 430529
Use this number as the identifier for your KIP and increment this value.
Adopted KIPs
Please insert new rows in sorted order (descending by KIP number).
KIP (please keep this sorted by KIP number) | Release | |||||||
---|---|---|---|---|---|---|---|---|
1 | KIP- | 425: Add some Log4J Kafka Appender Properties for Producing to500: Replace ZooKeeper with a Self-Managed Metadata Quorum | ||||||
2 | Secured Brokers2.3.0 | 2KIP- | 420: Add Single Value Fetch in Session Stores482: The Kafka Protocol should Support Optional Tagged Fields | 2. | 24.0 (WIP) | |||
3 | KIP-414: Expose Embedded ClientIds in Kafka Streams504 - Add new Java Authorizer Interface | 2.24.0 (WIP) | ||||||
4 | KIP-402: Improve fairness in SocketServer processors503: Add metric for number of topics marked for deletion | 2.24.0 (WIP) | ||||||
5 | KIP-394: Require member.id for initial join group request497: Add inter-broker API to alter ISR | 2.24.0 (WIP) | ||||||
6 | KIP-393: Time windowed serde to properly deserialize changelog input topic496: Administrative API to delete consumer offsets | 2.24.0 (WIP) | ||||||
7 | KIP- | 386: Standardize on Min/Avg/Max metrics' default value495: Dynamically Adjust Log Levels in Connect | 2. | 24.0 (WIP) | ||||
8 | KIP-382: MirrorMaker 2.0488: Clean up Sum,Count,Total Metrics | 2.34.0 (WIP) | ||||||
9 | KIP-380: Detect outdated control requests and bounced brokers using broker generation484: Expose metrics for group and transaction metadata loading duration | 2.24.0 (WIP) | ||||||
10 | KIP-377: TopicCommand to use AdminClient481: SerDe Improvements for Connect Decimal type in JSON | 2.24.0 (WIP) | ||||||
11 | KIP- | 376: Implement AutoClosable on appropriate classes that want to be used in a try-with-resource statement480: Sticky Partitioner | 2. | 24.0 | ||||
12 | KIP- | 372: Naming Repartition Topics for Joins and Grouping479: Add Materialized to Join | 2. | 14.0(WIP) | ||||
13 | KIP- | 371476: Add | a configuration to build custom SSL principal nameJava AdminClient Interface | 2. | 24.0 | |||
14 | KIP 368: Allow SASL Connections to Periodically Re-Authenticate-475: New Metrics to Measure Number of Tasks on a Connector | 2.24.0 (WIP) | ||||||
15 | KIP- | 367 Introduce close(Duration) to Producer and AdminClient instead of close(long, TimeUnit474: To deprecate WindowStore#put(key, value) | 2. | 24.0 (WIP) | ||||
16 | KIP-366: Make FunctionConversions deprecated471: Expose RocksDB Metrics in Kafka Streams | 2. 14.0 (WIP) | ||||||
17 | KIP-365: Materialized, Serialized, Joined, Consumed and Produced with implicit Serde467: Augment ProduceResponse error messaging for specific culprit records | 2.14.0 (WIP) | ||||||
18 | KIPKIP-361465: Add Consumer Configuration to Disable Auto Topic CreationConsolidated Connector Endpoint to Connect REST API | 2.3.0 (WIP) | ||||||
19 | KIP-359: Verify leader epoch in produce requests464: Defaults for AdminClient#createTopic | 2.34.0 (WIP) | ||||||
20 | KIP-358: Migrate Streams API to Duration instead of long ms times462: Use local thread id for KStreams | 2.13.0 | ||||||
21 | KIP- | 357: Add support to list ACLs per principal461: Improve Replica Fetcher behavior at handling partition failure | 2. | 13.0 | ||||
22 | KIP- | 356: Add withCachingDisabled() to StoreBuilder460: Admin Leader Election RPC | 2. | 14.0 | ||||
23 | KIP- | 353: Improve Kafka Streams Timestamp Synchronization458: Connector Client Config Override Policy | 2. | 13.0 | ||||
24 | KIP-351: Add --under-min-isr option to describe topics command455: Create an Administrative API for Replica Reassignment | 2.34.0 (WIP) | ||||||
25 | KIP-346: Improve LogCleaner behavior on error454: Expansion of the ConnectClusterState interface | 2.3.10 | ||||||
26 | KIP-342 Add support for custom SASL extensions in OAuthBearer authentication453: Add close() method to RocksDBConfigSetter | 2.13.0 | ||||||
27 | KIP- | 341: Update Sticky Assignor's User Data Protocol449: Add connector contexts to log messages in Connect workers | 2.3.0 | (WIP)|||||
28 | KIP-340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file446: Add changelog topic configuration to KTable suppress | 2.14.0 (WIP) | ||||||
29 | KIP-339: Create a new IncrementalAlterConfigs API445: In-memory Session Store | 2.3.0 (WIP) | ||||||
30 | 338 Support to exclude the internal topics in kafka-topics.sh command | 2.13.0 | ||||||
31 | KIP-336: Consolidate ExtendedSerializer/Serializer and ExtendedDeserializer/Deserializer442: Return to default max poll interval in Streams | 2.13.0 | ||||||
32 | KIP- | 332: Update AclCommand to use AdminClient API440: Extend Connect Converter to support headers | 2. | 14.0 (WIP) | ||||
33 | KIP-331 Add default implementation to close() and configure() for Serializer, Deserializer and Serde436: Add a metric indicating start time | 2.3.0 (WIP) | ||||||
34 | KIP-330434: Add retentionPeriod in SessionBytesStoreSupplierReplica Fetcher and Log Cleaner Count Metrics | 2.14.0 (WIP) | ||||||
35 | KIP-328: Ability to suppress updates for KTables430 - Return Authorized Operations in Describe Responses | 2.1.0 (partially implemented) / 2.33.0 | ||||||
36 | KIP-429: Kafka Consumer Incremental Rebalance Protocol | 2.4.0 (WIP) | ||||||
3637 | KIP- | 324428: Add | method to get metrics() in AdminClientin-memory window store | 2. | 13.0 | |||
3738 | KIP-322: Return new error code for DeleteTopics API when topic deletion disabled.427: Add AtMinIsr topic partition category (new metric & TopicCommand option) | 2.13.0 | 38||||||
39 | KIP-321: Update TopologyDescription to better represent Source and Sink Nodes425: Add some Log4J Kafka Appender Properties for Producing to Secured Brokers | 2.13.0 | ||||||
3940 | KIP-320: Allow fetchers to detect and handle log truncation421: Support resolving externalized secrets in AbstractConfig | 2.13.0 (partially implemented) / 2.2.0 | ||||||
4140 | KIP- | 319: Replace segments with segmentInterval in WindowBytesStoreSupplier420: Add Single Value Fetch in Session Stores | 2. | 12.0 | 41||||
42 | KIP- | 313: Add KStream.flatTransform and KStream.flatTransformValues2.2.0 (partially implemented) / 2.3.0 (WIP) | ||||||
42 | KIP-312 Add Overloaded StreamsBuilder Build Method to Accept java.util.Properties | 2.1.0 | ||||||
43 | KIP-308: Support dynamic update of max.connections.per.ip/max.connections.per.ip.overrides | 2.1.0 | ||||||
44 | KIP-306: Configuration for Delaying Response to Failed Authentication | 2.1.0 | ||||||
45 | KIP-305: Add Connect primitive number converters | 2.0.0 | ||||||
46 | KIP-303: Add Dynamic Routing in Streams Sink | 2.0.0 | ||||||
417: Allow JmxTool to connect to a secured RMI port | 2.3.0 | |||||||
43 | KIP-415: Incremental Cooperative Rebalancing in Kafka Connect | 2.3.0 | ||||||
44 | KIP-414: Expose Embedded ClientIds in Kafka Streams | 2.2.0 | ||||||
45 | KIP-412: Extend Admin API to support dynamic application log levels | 2.4.0 (WIP) | ||||||
46 | 2.3.0 | |||||||
47 | KIP-402: Improve fairness in SocketServer processors | 2.2.0 (partially implemented) / 2.3 | 47 | KIP-302 - Enable Kafka clients to use all DNS resolved IP addresses | 2.1.0 | |||
48 | KIP-298: Error Handling in Connect401: TransformerSupplier/ProcessorSupplier StateStore connecting | 2.04.0 (WIP) | ||||||
49 | KIP-297: Externalizing Secrets for Connect Configurations396: Add Commit/List Offsets Operations to AdminClient | 2.04.0 (WIP) | ||||||
50 | KIP-295 Add Streams Configuration Allowing for Optional Topology Optimization394: Require member.id for initial join group request | 2.02.0 | ||||||
51 | KIP-294 - Enable TLS hostname verification by default393: Time windowed serde to properly deserialize changelog input topic | 2.02.0 | ||||||
52 | KIP- | 292: Add transformValues() method to KTable389: Introduce a configurable consumer group size limit | 2. | 02.0 | ||||
53 | KIP-290: Support for Prefixed ACLs386: Standardize on Min/Avg/Max metrics' default value | 2.02.0 | ||||||
54 | KIP-289: Improve the default group id behavior in KafkaConsumer382: MirrorMaker 2.0 | 2.24.0 | ||||||
55 | 285: Connect Rest Extension Plugin | 2.04.0 (WIP) | ||||||
56 | KIP-284: Set default retention ms for Streams repartition topics to Long.MAX_VALUE380: Detect outdated control requests and bounced brokers using broker generation | 2.02.0 | ||||||
57 | KIP-283: Efficient Memory Usage for Down-Conversion377: TopicCommand to use AdminClient | 2.02.0 | ||||||
58 | KIP-282: Add the listener name to the authentication context376: Implement AutoClosable on appropriate classes that want to be used in a try-with-resource statement | 2.02.0 | ||||||
59 | KIP-281: ConsumerPerformance: Increase Polling Loop Timeout and Make It Reachable by the End User374: Add '--help' option to all available Kafka CLI commands | 2.02.0 | ||||||
60 | KIP- | 279: Fix log divergence between leader and follower after fast leader fail over372: Naming Repartition Topics for Joins and Grouping | 2. | 01.0 | ||||
61 | KIP- | 278 - Add version option to Kafka's commands371: Add a configuration to build custom SSL principal name | 2. | 02.0 | ||||
62 | KIP- | 277 - Fine Grained ACL for CreateTopics API369: Alternative Partitioner to Support "Always Round-Robin" Selection | 2. | 04.0 | ||||
63 | KIP -276 - Add StreamsConfig prefix for different consumers368: Allow SASL Connections to Periodically Re-Authenticate | 2.02.0 | ||||||
64 | KIP- | 274: Kafka Streams Skipped Records Metrics367 Introduce close(Duration) to Producer and AdminClient instead of close(long, TimeUnit) | 2. | 02.0 | ||||
65 | KIP- | 272: Add API version tag to broker's RequestsPerSec metric366: Make FunctionConversions deprecated | 2. | 01.0 | ||||
66 | KIP-270 - A Scala Wrapper Library for Kafka Streams365: Materialized, Serialized, Joined, Consumed and Produced with implicit Serde | 2.01.0 | ||||||
67 | KIPKIP- | 268: Simplify Kafka Streams Rebalance Metadata Upgrade361: Add Consumer Configuration to Disable Auto Topic Creation | 2. | 03.0 | ||||
68 | KIP- | 267: Add Processor Unit Test Support to Kafka Streams Test Utils360: Improve handling of unknown producer | 2. | 04.0 (WIP) | ||||
69 | KIP-266: Fix consumer indefinite blocking behavior359: Verify leader epoch in produce requests | 2.04.0 (WIP) | ||||||
70 | KIP- | 265: Make Windowed Serde to public APIs358: Migrate Streams API to Duration instead of long ms times | 2. | 01.0 | ||||
71 | KIP-261357: Add Single Value Fetch in Window Storessupport to list ACLs per principal | 2.01.0 | ||||||
72 | KIP-258: Allow to Store Record Timestamps in RocksDB356: Add withCachingDisabled() to StoreBuilder | 2.31.0 | ||||||
73 | KIP- | 257 - Configurable Quota Management354: Add a Maximum Log Compaction Lag | 2. | 03.0 | ||||
74 | KIP-255: OAuth Authentication via SASL/OAUTHBEARER353: Improve Kafka Streams Timestamp Synchronization | 2.01.0 | ||||||
75 | KIP- | 251: Allow timestamp manipulation in Processor API352: Distinguish URPs caused by reassignment | 2. | 04.0 (WIP) | ||||
76 | KIP- | 249351: Add | Delegation Token Operations to KafkaAdminClient--under-min-isr option to describe topics command | 2. | 03.0 | |||
77 | KIP- | 247: Add public test utils for Kafka Streams346: Improve LogCleaner behavior on error | 2.1 | 1.1.0|||||
78 | KIP- | 245: Use Properties instead of StreamsConfig in KafkaStreams constructor345: Introduce static membership protocol to reduce consumer rebalances | 2.4.0 (WIP), partially available in 2.3.0 | |||||
79 | KIP-244: 342 Add Record Header support to Kafka Streams Processor APIsupport for custom SASL extensions in OAuthBearer authentication | 2.01.0 | ||||||
80 | KIP- | 243: Make ProducerConfig and ConsumerConfig constructors public341: Update Sticky Assignor's User Data Protocol | 2.3 | 1.1.0 | ||||
81 | KIP | -239 Add queryableStoreName() to GlobalKTable-340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file | 2 | 1.1.0 | ||||
82 | KIP- | 238: Expose Kafka cluster ID in Connect REST API339: Create a new IncrementalAlterConfigs API | 2.3.0 | 1.1.0|||||
83 | KIP- | 237: More Controller Health Metrics338 Support to exclude the internal topics in kafka-topics.sh command | 2. | 01.0 | ||||
84 | KIP-235: Add DNS alias support for secured connection336: Consolidate ExtendedSerializer/Serializer and ExtendedDeserializer/Deserializer | 2.1.0 | ||||||
85 | KIP-233: Simplify StreamsBuilder#addGlobalStore332: Update AclCommand to use AdminClient API | 21.1.0 | ||||||
86 | KIP-231: Improve the Required ACL of ListGroups API331 Add default implementation to close() and configure() for Serializer, Deserializer and Serde | 2.13.0 | ||||||
87 | KIP- | 229: DeleteGroups API330: Add retentionPeriod in SessionBytesStoreSupplier | 2 | 1.1.0 | ||||
88 | KIP- | 227 - Introduce Incremental FetchRequests to Increase Partition Scalability328: Ability to suppress updates for KTables | 2 | 1.1.0 | ||||
89 | KIP-226 - Dynamic Broker Configuration | 1.1.0 | ||||||
(partially implemented) / 2.3.0 (WIP) | ||||||||
89 | KIP-324: Add method to get metrics() in AdminClient | 2 | 90 | KIP-225 - Use tags for consumer “records.lag” metrics | 1.1.0 | |||
9190 | KIP-224: Add configuration parameter `retries` to Streams API322: Return new error code for DeleteTopics API when topic deletion disabled. | 21.1.0 | ||||||
9291 | KIP- | 223 - Add per-topic min lead and per-partition lead metrics to KafkaConsumer321: Update TopologyDescription to better represent Source and Sink Nodes | 2. | 01.0 | ||||
92 | KIP- | 222 - Add Consumer Group operations to Admin API320: Allow fetchers to detect and handle log truncation | 2.1.0 (partially implemented) / 2.2.0 | |||||
9493 | KIP- | 220: Add AdminClient into Kafka Streams' ClientSupplier319: Replace segments with segmentInterval in WindowBytesStoreSupplier | 2 | 1.1.0 | 95||||
94 | KIP- | 219 - Improve quota communication313: Add KStream.flatTransform and KStream.flatTransformValues | 2. | 02.0 | 96 | KIP-218: Make KafkaFuture.Function java 8 lambda compatible | (partially implemented) / 2.3 | 1.1.0 |
95 | KIP- | 215: Add topic regex support for Connect sinks312 Add Overloaded StreamsBuilder Build Method to Accept java.util.Properties | 2 | 1.1.0 | ||||
9896 | KIP-214: Add zookeeper.max.in.flight.requests config to the broker308: Support dynamic update of max.connections.per.ip/max.connections.per.ip.overrides | 21.1.0 | ||||||
97 | KIP- | 212: Enforce set of legal characters for connector names307: Allow to define custom processor names with KStreams DSL | 2.3.0 (partial) | |||||
98 | KIP-306: Configuration for Delaying Response to Failed Authentication | 1.1.0 | 100 | KIP-211: Revise Expiration Semantics of Consumer Group Offsets2.1.0 | ||||
10199 | KIP-210 - Provide for custom error handling when Kafka Streams fails to produce305: Add Connect primitive number converters | 2.01.1.0 | ||||||
102100 | KIP- | 208303: Add | SSL support to Kafka Connect REST interfaceDynamic Routing in Streams Sink | 2.0 | 1.1.0 | |||
101 | KIP- | 207: Offsets returned by ListOffsetsResponse should be monotonically increasing even during a partition leader change302 - Enable Kafka clients to use all DNS resolved IP addresses | 2. | 21.0 | ||||
102 | KIP-206300: Add support for UUID serialization and deserializationWindowed KTable API in StreamsBuilder | 2.14.0 | ||||||
105103 | KIP- | 205: Add all() and range() API to ReadOnlyWindowStore298: Error Handling in Connect | 2.0 | 1.1.0 | ||||
106104 | KIP- | 204 : Adding records deletion operation to the new Admin Client API297: Externalizing Secrets for Connect Configurations | 2.0 | 1.1.0 | ||||
107105 | KIP-295 Add Streams Configuration Allowing for Optional Topology Optimization | 2.0.0 | ||||||
106 | KIP-294 - Enable TLS hostname verification by default | 2.0.0 | ||||||
107 | Accepted | |||||||
1.1.0 | 108 | KIP- | 202292: | Move mergeAdd transformValues() | from StreamsBuildermethod to | KStreamKTable | 12.0.0 | |
109 | KIP- | 198: Remove ZK dependency from Streams Reset Tool290: Support for Prefixed ACLs | 2 | 1.0.0 | ||||
110 | KIP- | 197 Connect REST API should include the connector type when describing a connector289: Improve the default group id behavior in KafkaConsumer | 2.2 | 1.0.0 | ||||
111 | KIP-196285: Add metrics to Kafka Connect frameworkConnect Rest Extension Plugin | 21.0.0 | ||||||
112 | KIP- | 195: AdminClient.createPartitions284: Set default retention ms for Streams repartition topics to Long.MAX_VALUE | 2 | 1.0.0 | ||||
113 | KIP- | 192 : Provide cleaner semantics when idempotence is enabled283: Efficient Memory Usage for Down-Conversion | 2 | 1.0.0 | ||||
114 | KIP- | 191: KafkaConsumer.subscribe() overload that takes just Pattern282: Add the listener name to the authentication context | 2 | 1.0.0 | ||||
115 | KIP- | 190: Handle client-ids consistently between clients and brokers281: ConsumerPerformance: Increase Polling Loop Timeout and Make It Reachable by the End User | 2 | 1.0.0 | ||||
116 | KIP-189: Improve principal builder interface and add support for SASL279: Fix log divergence between leader and follower after fast leader fail over | 21.0.0 | ||||||
117 | KIP- | 188278 - Add | new metrics to support health checksversion option to Kafka's commands | 2 | 1.0.0 | |||
118 | KIP- | 187 - Add cumulative count metric for all Kafka rate metrics277 - Fine Grained ACL for CreateTopics API | 2 | 1.0.0 | ||||
119 | KIP-186: Increase offsets retention default to 7 days276 - Add StreamsConfig prefix for different consumers | 2.0.0 | ||||||
120 | KIP-183 - Change PreferredReplicaLeaderElectionCommand to use AdminClient274: Kafka Streams Skipped Records Metrics | 2.20.0 | ||||||
121 | KIP- | 182: Reduce Streams DSL overloads and allow easier use of custom storage engines272: Add API version tag to broker's RequestsPerSec metric | 2 | 1.0.0 | ||||
122 | KIP- | 180: Add a broker metric specifying the number of consumer group rebalances in progress270 - A Scala Wrapper Library for Kafka Streams | 2.0 | 1.1.0 | ||||
123 | KIP-177: Consumer perf tool should count rebalance time268: Simplify Kafka Streams Rebalance Metadata Upgrade | 21.0.0 | ||||||
124 | KIP- | 176: Remove deprecated new-consumer option for tools267: Add Processor Unit Test Support to Kafka Streams Test Utils | 2.0.0 | |||||
125 | KIP- | 175: Additional '--describe' views for ConsumerGroupCommand266: Fix consumer indefinite blocking behavior | 2.0 | 1.1.0 | ||||
126 | KIP-174 - Deprecate and remove internal converter configs in WorkerConfig265: Make Windowed Serde to public APIs | 2.0.0 | ||||||
127 | KIP-173: Add prefix to StreamsConfig to enable setting default internal topic configs261: Add Single Value Fetch in Window Stores | 21.0.0 | ||||||
128 | KIP- | 171 - Extend Consumer Group Reset Offset for Stream Application258: Allow to Store Record Timestamps in RocksDB | 2.3.0 (partially implemented) | 1.1.0|||||
129 | KIP-168: Add GlobalTopicCount and GlobalPartitionCount metric per cluster-257 - Configurable Quota Management | 21.0.0 | ||||||
130 | KIP- | 167: Add interface for the state store restoration process255: OAuth Authentication via SASL/OAUTHBEARER | 2 | 1.0.0 | ||||
131 | KIP-164 - Add UnderMinIsrPartitionCount and per-partition UnderMinIsr metrics251: Allow timestamp manipulation in Processor API | 21.0.0 | ||||||
132 | KIP- | 163: Lower the Minimum Required ACL Permission of OffsetFetch249: Add Delegation Token Operations to KafkaAdminClient | 2 | 1.0.0 | ||||
133 | KIP-162: Enable topic deletion by default247: Add public test utils for Kafka Streams | 1.01.0 | ||||||
134 | KIP-161: streams deserialization exception handlers245: Use Properties instead of StreamsConfig in KafkaStreams constructor | 21.0.0 | ||||||
135 | KIP- | 160: Augment KStream.print(), KStream.writeAsText() to allow users pass in extra parameters in the printed string244: Add Record Header support to Kafka Streams Processor API | 2 | 1.0.0 | ||||
136 | KIP- | 157 - Add consumer config options to streams reset tool243: Make ProducerConfig and ConsumerConfig constructors public | 1. | 01.0 | ||||
137 | KIP- | 156 Add option "dry run" to Streams application reset tool239 Add queryableStoreName() to GlobalKTable | 1.1.0 | 0.11.0.0|||||
138 | KIP- | 155 - Add range scan for windowed state stores238: Expose Kafka cluster ID in Connect REST API | 1.1 | 0.11.0.0 | ||||
139 | KIP-154 Add Kafka Connect configuration properties for creating internal topics237: More Controller Health Metrics | 2.0.11.0.0 | ||||||
140 | KIP- | 153: Include only client traffic in BytesOutPerSec metric235: Add DNS alias support for secured connection | 2.1 | 0.11.0.0 | ||||
141 | KIP-152 - Improve diagnostics for SASL authentication failures233: Simplify StreamsBuilder#addGlobalStore | 1.01.0 | ||||||
142 | KIP- | 151 Expose Connector type in REST231: Improve the Required ACL of ListGroups API | 02. | 111.0 | .0||||
143 | KIP-150 - Kafka-Streams Cogroup229: DeleteGroups API | 1.1.02.3.0 (WIP) | ||||||
144 | KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner-227 - Introduce Incremental FetchRequests to Increase Partition Scalability | 1.1.0 (partially implemented) (WIP for 2.3.0) | ||||||
145 | KIP-146 - Classloading Isolation in Connect226 - Dynamic Broker Configuration | 1.10.11.0.0 | ||||||
146 | KIP-145 - Expose Record Headers in Kafka Connect225 - Use tags for consumer “records.lag” metrics | 1.1.0 | ||||||
147 | KIP-144: Exponential backoff for broker reconnect attempts224: Add configuration parameter `retries` to Streams API | 1.10.11.0.0 | ||||||
148 | KIP-143: Controller Health Metrics-223 - Add per-topic min lead and per-partition lead metrics to KafkaConsumer | 2.0.00.11.0.0 | ||||||
149 | KIP-222 - 140: Add administrative RPCs for adding, deleting, and listing ACLsAdd Consumer Group operations to Admin API | 20.11.0.0 | ||||||
150 | KIP-138: Change punctuate semantics221: Enhance DSL with Connecting Topic Creation and Repartition Hint | 2.4.0 (WIP)1.0.0 | ||||||
151 | KIP-137: Enhance TopicCommand --describe to show topics marked for deletion220: Add AdminClient into Kafka Streams' ClientSupplier | 1.10.11.0.0 | ||||||
152 | KIP-136: Add Listener name to SelectorMetrics tags219 - Improve quota communication | 2.0.11.0.0 | ||||||
153 | KIP-134: Delay initial consumer group rebalance218: Make KafkaFuture.Function java 8 lambda compatible | 1.1.00.11.0.0 | ||||||
154 | KIP-133: Describe and Alter Configs Admin APIs215: Add topic regex support for Connect sinks | 1.1.00.11.0.0 | ||||||
155 | KIP- | 130: Expose states of active tasks to KafkaStreams public API214: Add zookeeper.max.in.flight.requests config to the broker | 1. | 01.0 | ||||
156 | KIP- | 129: Streams Exactly-Once Semantics213 Support non-key joining in KTable | 2.4.0 (WIP) | 0.11.0.0|||||
157 | KIP-128: Add ByteArrayConverter for Kafka Connect212: Enforce set of legal characters for connector names | 1.1.00.11.0.0 | ||||||
158 | KIP-126 - Allow KafkaProducer to split and resend oversized batches.211: Revise Expiration Semantics of Consumer Group Offsets | 2.10.11.0.0 | ||||||
159 | KIP-124 - Request rate quotas210 - Provide for custom error handling when Kafka Streams fails to produce | 1.10.11.0.0 | ||||||
160 | KIP-123: Allow per stream/table timestamp extractor208: Add SSL support to Kafka Connect REST interface | 1.1.00.11.0.0 | ||||||
161 | KIP-122207: Add Reset Consumer Group Offsets toolingOffsets returned by ListOffsetsResponse should be monotonically increasing even during a partition leader change | 2.2.00.11.0.0 | ||||||
162 | KIP- | 121206: Add | KStream peek method0.11.0.0support for UUID serialization and deserialization | 2.1.0 | ||||
163 | KIP-120: Cleanup Kafka Streams builder API205: Add all() and range() API to ReadOnlyWindowStore | 1.01.0 | ||||||
164 | KIP-204 : Adding records deletion operation to the new Admin Client API | 1.1119: Drop Support for Scala 2.10 in Kafka 0.110.11.0.0 | ||||||
165 | KIP-118: Drop Support for Java 7203: Add toLowerCase support to sasl.kerberos.principal.to.local rule | 1.12.0.0 | ||||||
166 | KIP-117: Add a public AdminClient API for Kafka admin operations202: Move merge() from StreamsBuilder to KStream | 1.0.11.0.0 | ||||||
167 | KIP-115: Enforce offsets.topic.replication.factor upon __consumer_offsets 198: Remove ZK dependency from Streams Reset Tool | 1 auto topic creation0.11.0.0 | ||||||
168 | KIP-114: KTable state stores and improved semantics197 Connect REST API should include the connector type when describing a connector | 1.0.11.0.0 | ||||||
169 | KIP-113: Support replicas movement between log directories196: Add metrics to Kafka Connect framework | 1.10.0 | ||||||
170 | KIP-112: Handle disk failure for JBOD195: AdminClient.createPartitions | 1.0.0 | ||||||
171 | KIP- | 110: Add Codec for ZStandard Compression192 : Provide cleaner semantics when idempotence is enabled | 1.0 | 2.1.0 | ||||
172 | KIP-109: Old Consumer Deprecation191: KafkaConsumer.subscribe() overload that takes just Pattern | 1.0.11.0.0 | ||||||
173 | KIP-108: Create Topic Policy190: Handle client-ids consistently between clients and brokers | 1.0.0.10.2.0 | ||||||
174 | KIP-107: Add deleteRecordsBefore() API in AdminClient189: Improve principal builder interface and add support for SASL | 1.0.11.0.0 | ||||||
175 | KIP-106 - Change Default unclean.leader.election.enabled from True to False188 - Add new metrics to support health checks | 10.11.0.0 | ||||||
176 | KIP-105: Addition of Record Level for Sensors-187 - Add cumulative count metric for all Kafka rate metrics | 1.0.0.10.2.0 | ||||||
177 | KIP-104: Granular Sensors for Streams 186: Increase offsets retention default to 7 days | 2.00.10.2.0 | ||||||
178 | KIP-103: Separation of Internal and External traffic183 - Change PreferredReplicaLeaderElectionCommand to use AdminClient | 2.0.10.2.0 | ||||||
179 | KIP-102 - Add close with timeout for consumers182: Reduce Streams DSL overloads and allow easier use of custom storage engines | 1.0.0.10.2.0 | ||||||
180 | KIP-101 - Alter Replication Protocol to use Leader Epoch rather than High Watermark for Truncation180: Add a broker metric specifying the number of consumer group rebalances in progress | 1.10.11.0.0 | ||||||
181 | KIP-100 - Relax Type constraints in Kafka Streams API177: Consumer perf tool should count rebalance time | 1.0.0.10.2.0 | ||||||
182 | KIP- | 99: Add Global Tables to Kafka Streams176: Remove deprecated new-consumer option for tools | 2.0 | 0.10.2.0 | ||||
183 | KIP-98 - Exactly Once Delivery and Transactional Messaging175: Additional '--describe' views for ConsumerGroupCommand | 1.10.11.0.0 | ||||||
184 | KIP-97: Improved Kafka Client RPC Compatibility Policy-174 - Deprecate and remove internal converter configs in WorkerConfig | 2.0.0.10.2.0 | ||||||
185 | KIP-96 - Add per partition metrics for in-sync and assigned replica count173: Add prefix to StreamsConfig to enable setting default internal topic configs | 1.0.0.10.2.0 | ||||||
186 | KIP-94 Session Windows171 - Extend Consumer Group Reset Offset for Stream Application | 1.10.10.2.0 | ||||||
187 | KIP-93: Improve invalid timestamp handling in Kafka Streams168: Add GlobalTopicCount and GlobalPartitionCount metric per cluster | 1.00.10.2.0 | ||||||
188 | KIP-92 - Add per partition lag metrics to KafkaConsumer167: Add interface for the state store restoration process | 1.0.0.10.2.0 | ||||||
189 | -91 Provide Intuitive User Timeouts in The Producer -164 - Add UnderMinIsrPartitionCount and per-partition UnderMinIsr metrics | 1.02.1.0 | ||||||
190 | 90 - Remove zkClient dependency from Streams | 1.0 | 0.10.2.0 | |||||
191 | KIP-89: Allow sink connectors to decouple flush and offset commit162: Enable topic deletion by default | 1.0.0.10.2.0 | ||||||
192 | KIP-88: OffsetFetch Protocol Update161: streams deserialization exception handlers | 1.0.0.10.2.0 | ||||||
193 | KIP-86: Configurable SASL callback handlers160: Augment KStream.print(), KStream.writeAsText() to allow users pass in extra parameters in the printed string | 12.0.0 | ||||||
194 | KIP- | 85: Dynamic JAAS configuration for Kafka clients157 - Add consumer config options to streams reset tool | 1.0. | 0.10.2.0 | ||||
195 | KIP- | 84: Support SASL SCRAM mechanisms156 Add option "dry run" to Streams application reset tool | 0. | 1011. | 20.0 | |||
196 | KIP- | 82155 - Add | Record Headersrange scan for windowed state stores | 0.11.0.0 | ||||
197 | KIP-81: Bound Fetch memory usage in the consumer154 Add Kafka Connect configuration properties for creating internal topics | 0.11.0.02.2.0 (WIP) | ||||||
198 | KIP-79 - ListOffsetRequest/ListOffsetResponse v1 and add timestamp search methods to the new consumer153: Include only client traffic in BytesOutPerSec metric | 0.1011.10.0 | ||||||
199 | KIP-78: Cluster Id-152 - Improve diagnostics for SASL authentication failures | 1.00.10.1.0 | ||||||
200 | KIP-77: Improve Kafka Streams Join Semantics151 Expose Connector type in REST API | 0.1011.20.0 | ||||||
201 | KIP-75 150 - Add per-connector ConvertersKafka-Streams Cogroup | 2.4.0 (WIP)0.10.1.0 | ||||||
202 | KIP-74: Add Fetch Response Size Limit in Bytes149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner | 1.1.0 (partially implemented) (WIP for 2.4.0)0.10.1.0 | ||||||
203 | KIP-73: Replication Quotas-146 - Classloading Isolation in Connect | 0. 1011. 10.0 | ||||||
204 | KIP-72: Allow putting a bound on memory consumed by Incoming request -145 - Expose Record Headers in Kafka Connect | 1.01.0 | ||||||
205 | KIP-71: Enable log compaction and deletion to co-exist144: Exponential backoff for broker reconnect attempts | 0.1011.10.0 | ||||||
206 | KIP-70: Revise Partition Assignment Semantics on New Consumer's Subscription Change143: Controller Health Metrics | 0.1011.10.0 | ||||||
207 | KIP-67: Queryable state for Kafka Streams140: Add administrative RPCs for adding, deleting, and listing ACLs | 0.1011.10.0 | ||||||
208 | KIP-66: Single Message Transforms for Kafka Connect138: Change punctuate semantics | 10.10.2.0 / 0.11.0.0 | ||||||
209 | KIP-65: Expose timestamps to Connect137: Enhance TopicCommand --describe to show topics marked for deletion | 0.1011.10.0 | ||||||
210 | KIP-63: Unify store and downstream caching in streams136: Add Listener name to SelectorMetrics tags | 0.1011.10.0 | ||||||
211 | KIP-62134: Allow consumer to send heartbeats from a background threadDelay initial consumer group rebalance | 0.1011.10.0 | ||||||
212 | KIP-60 - Make Java client classloading more flexible133: Describe and Alter Configs Admin APIs | 0.1011.10.0 | ||||||
213 | KIP- | 58 - Make Log Compaction Point Configurable130: Expose states of active tasks to KafkaStreams public API | 1.0. | 0.10.1.0 | ||||
214 | KIP-57 - Interoperable LZ4 Framing129: Streams Exactly-Once Semantics | 0.1011.0.0 | ||||||
215 | KIP-56: Allow cross origin HTTP requests on all HTTP methods128: Add ByteArrayConverter for Kafka Connect | 0.1011.0.0 | ||||||
216 | KIP-55: Secure Quotas for Authenticated Users-126 - Allow KafkaProducer to split and resend oversized batches. | 0.11.00.10.1.0 | ||||||
217 | KIP-54: Sticky Partition Assignment Strategy124 - Request rate quotas | 0.11.0.0 | ||||||
218 | KIP-52: Connector Control APIs123: Allow per stream/table timestamp extractor | 0.1011.0.0 | ||||||
219 | KIP-51 - List Connectors REST API122: Add Reset Consumer Group Offsets tooling | 0.1011.0.0 | ||||||
220 | KIP-50 - Move Authorizer to o.a.k.common package121: Add KStream peek method | 0. 1011. 10.0 | ||||||
221 | KIP-48 Delegation token support for Kafka120: Cleanup Kafka Streams builder API | 1.10.0 | ||||||
222 | KIP-45 - Standardize all client sequence interaction on j.u.Collection.119: Drop Support for Scala 2.10 in Kafka 0.11 | 0.1011.0.0 | ||||||
223 | KIP-43: Kafka SASL enhancements118: Drop Support for Java 7 | 2.0.10.0.0 | ||||||
224 | KIP-42117: Add Producer and Consumer Interceptorsa public AdminClient API for Kafka admin operations | 0.1011.0.0 | ||||||
225 | KIP-41: Consumer Max Records115: Enforce offsets.topic.replication.factor upon __consumer_offsets auto topic creation | 0.1011.0.0 | ||||||
226 | KIP-40114: ListGroups and DescribeGroupKTable state stores and improved semantics | 0.911.0.0 | ||||||
227 | KIP-38: ZooKeeper Authentication113: Support replicas movement between log directories | 1.10.9.0.0 | ||||||
228 | KIP-36 - Rack aware replica assignment112: Handle disk failure for JBOD | 10.10.0.0 | ||||||
229 | KIP- | 35 - Retrieving protocol version110: Add Codec for ZStandard Compression | 2.1 | 0.10.0.0 | ||||
230 | KIP-33 - Add a time based log index109: Old Consumer Deprecation | 0.1011.10.0 | ||||||
231 | KIP-32 - Add timestamps to Kafka message108: Create Topic Policy | 0.10.02.0 | ||||||
232 | KIP-31 - Move to relative offsets in compressed message sets107: Add deleteRecordsBefore() API in AdminClient | 0.1011.0.0 | ||||||
233 | 28 - Add a processor client 106 - Change Default unclean.leader.election.enabled from True to False | 0.1011.0.0 | ||||||
234 | KIP-26 - Add Kafka Connect framework for data import/export105: Addition of Record Level for Sensors | 0.910.02.0 | ||||||
235 | KIP-25 - System test improvements104: Granular Sensors for Streams | 0.910.02.0 | ||||||
236 | KIP-22 - Expose a Partitioner interface in the new producer103: Separation of Internal and External traffic | 0.910.02.0 | ||||||
237 | KIP-21 - Dynamic Configuration102 - Add close with timeout for consumers | 0.910.02.0 (WIP) | ||||||
238 | KIP-20 Enable log preallocate to improve consume performance under windows and some old Linux file system-101 - Alter Replication Protocol to use Leader Epoch rather than High Watermark for Truncation | 0.911.0.0 | ||||||
239 | KIP-19 - Add a request timeout to NetworkClient100 - Relax Type constraints in Kafka Streams API | 0.910.02.0 | ||||||
240 | KIP-16 - Automated Replica Lag Tuning99: Add Global Tables to Kafka Streams | 0.910.02.0 | ||||||
241 | KIP-15 - Add a close method with a timeout in the producer98 - Exactly Once Delivery and Transactional Messaging | 0.911.0.0 | ||||||
242 | KIP-13 - Quota Design97: Improved Kafka Client RPC Compatibility Policy | 0.910.02.0 | ||||||
243 | KIP-12 - Kafka Sasl/Kerberos and SSL implementation96 - Add per partition metrics for in-sync and assigned replica count | 0.910.02.0 | ||||||
244 | KIP-11 - Kafka Authorizer design94 Session Windows | 0.910.02.0 | ||||||
245 | KIP-8 - Add a flush method to the producer API93: Improve invalid timestamp handling in Kafka Streams | 0.910.02.0 | ||||||
246 | KIP-4 - Metadata Protocol Changes92 - Add per partition lag metrics to KafkaConsumer | 0.10.02.0 | ||||||
247 | KIP-4 - Command line and centralized administrative operations91 Provide Intuitive User Timeouts in The Producer | 2.1.0 | ||||||
248 | KIP-90 - Remove zkClient dependency from Streams | 0.9.0.0, 0.10.02.0, | ||||||
249 | KIP-89: Allow sink connectors to decouple flush and offset commit | 0.10.12.0 | ||||||
248250 | KIP-3 - Mirror Maker Enhancement88: OffsetFetch Protocol Update | 0.910.02.0 | ||||||
249251 | KIP-86: Configurable SASL callback handlers | 2.0.0 | ||||||
252 | KIP-85: Dynamic JAAS configuration for Kafka clients - Refactor brokers to allow listening on multiple ports and IPs | 0.910.02.0 | ||||||
250253 | KIP-1 - Remove support of request.required.acks84: Support SASL SCRAM mechanisms | 0.910.02.0 |
...
...
KIPs under discussion
Please insert new rows in sorted order (ascending by KIP number). Please move the adopted/accepted KIPs to the "Adopted KIPs" table above.
Dormant/inactive KIPs
Please insert new rows in sorted order (ascending by KIP number).
Discarded KIPs
Please insert new rows in sorted order (ascending by KIP number).
Discarded KIPs
KIP | Comment | |
---|---|---|
KIP-5 - Broker Configuration Management | Superseded by KIP-21 | |
KIP-7 - Security - IP Filtering | ||
KIP-9 - SSL Support | Adopted via KAFKA-1690 but not via this KIP process | |
KIP-18 - JBOD Support | Superseded by KIP-112 and KIP-113 | |
KIP-24 - Remove ISR information from TopicMetadataRequest and add broker level metadata request | ||
KIP-29 - Add an IsrPropagateIntervalMs configuration to KafkaConfig | No longer need after KAFKA-2722 | |
KIP-34 Add Partitioner Change Listener to Partitioner Interface for Multiple Use Case | Per feedback it is better do this parition() method and avoid Thread Coordination etc. | |
KIP-80: Kafka Rest Server | ||
KIP-83 - Allow multiple SASL authenticated Java clients in a single JVM process | After KIP-85, no interface changes, reduced to KAFKA-4180 | |
KIP-116: Add State Store Checkpoint Interval Configuration | Currently not needed as checkpointing can be done on commit interval. | |
KIP-127: Pluggable JAAS LoginModule configuration for SSL | Similar functionality can be implemented using a custom PrincipalBuilder. | |
KIP-132: Augment KStream.print to allow extra parameters in the printed string | Duplicated by KIP-160 | |
KIP-184: Rename LogCleaner and related classes to LogCompactor | A lot of configuration change will cause more trouble and doesn't seems to be worth it | |
KIP-111: Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker. | Covered by KIP-189 | |
KIP-147: Add missing type parameters to StateStoreSupplier factories and KGroupedStream/Table methods | Covered by KIP-182 | |
KIP-165: Extend Interactive Queries for return latest update timestamp per key | Covered by KIP-258 | |
KIP-170: Enhanced TopicCreatePolicy and introduction of TopicDeletePolicy | Superseded by KIP-201: Rationalising Policy interfaces | |
KIP 172: Add regular-expression topic support for sink connector | Superseded by KIP-215: Add topic regex support for Connect sinks | KIP-179: Change ReassignPartitionsCommand to use AdminClient |
KIP 230: Name Windowing Joins | Covered by KIP-372: Naming Repartition Topics for Joins and Grouping | |
KIP-232: Detect outdated metadata using per-partition leaderEpoch field | Merged into KIP-320: Allow fetchers to detect and handle log truncation | |
KIP-246: Connect producers and consumers should inherit worker configs | Withdrawn because proposal could not be made backward compatible with existing behavior | |
KIP-262: Metadata should include number of state stores for task | Rejected since metadata is no longer required. | |
KIP-263: Allow broker to skip sanity check of inactive segments on broker startup | Withdrawn because the solution we agreed on does not require interface change | |
KIP-286: producer.send() should not block on metadata update | Withdrawn because the benefit of not having to wait for metadata is probably not worth the complexity added in producer. | |
KIP-288: [DISCARDED] Consumer.poll() timeout semantic change and new waitForAssignment method | Discarded in deference to KIP-266 | |
KIP-310: Add a Kafka Source Connector to Kafka Connect | Withdrawn in favor of KIP-382. | |
KIP-327: Add describe all topics API to AdminClient | Withdrawn in-favor of filter support in Metadata API and KIP-142 | |
KIP-344: The auto-generated client id should be passed to MetricsReporter | Bug fix so KIP not needed | |
KIP-364: Remove implicit Materialized, Consumed and Produced | Subsumed by KIP-365 | |
KIP-404: Add Kafka Connect configuration parameter for disabling WADL output on OPTIONS request | Discarded, reported as a bug by Jira | |
server | ASF JIRA | |
serverId | 5aa69414-a9e9-3523-82ec-879b028fb15b | key | KAFKA-7759
KIP Discussion Recordings
...