This page describes a proposed Kafka Improvement Proposal (KIP) process for proposing a major change to Kafka.
To create your own KIP, click on
Create from template | ||||||||
---|---|---|---|---|---|---|---|---|
|
dev@kafka.apache.org
and request permission (http://kafka.apache.org/contact). Also add an entry to the table KIPs under discussion (for Streams API KIPs, please also add it to Kafka Streams sub page)....
- Click
. Take the next available KIP number and give your proposal a descriptive heading. e.g. "KIP 42: Allow Infinite Retention With Bounded Disk Usage".Create from template templateName 54329345 templateId 54329345 title KIP-NEXT: Insert Title Here buttonLabel Create KIP - Fill in the sections as described above
- Start a [DISCUSS] thread on the Apache mailing list. Please ensure that the subject of the thread is of the format [DISCUSS] KIP-{your KIP number} {your KIP heading} The and the body contains a link to your new KIP. The discussion should happen on the mailing list, not on the wiki, since the wiki comment system doesn't work well for larger discussions. In the process of the discussion you may update the proposal. You should let people know the changes you are making. When you feel you have a finalized proposal
- Once the proposal is finalized call a [VOTE] to have the proposal adopted. These proposals are more serious than code changes and more serious even than release votes. The criteria for acceptance is lazy majority. The vote should remain open for at least 72 hours.
- Please update the KIP wiki page, and the index below, to reflect the current stage of the KIP after a vote. This acts as the permanent record indicating the result of the KIP (e.g., Accepted or Rejected). Also report the result of the KIP vote to the voting thread on the mailing list so the conclusion is clear.
KIP round-up
Next KIP Number: 388527
Use this number as the identifier for your KIP and increment this value.
Adopted KIPs
Please insert new rows in sorted order (descending by KIP number).
KIP (please keep this sorted by KIP number) | Release | ||||
---|---|---|---|---|---|
1 | KIP | -377: TopicCommand to use AdminClient-500: Replace ZooKeeper with a Self-Managed Metadata Quorum | |||
2 | KIP-482: The Kafka Protocol should Support Optional Tagged Fields | 2. | 24.0 (WIP) | ||
23 | KIP-504 - 331 Add default implementation to close() and configure() for Serializer, Deserializer and SerdeAdd new Java Authorizer Interface | 2.24.0 (WIP) | |||
34 | KIP-372: Naming Repartition Topics for Joins and Grouping503: Add metric for number of topics marked for deletion | 2.1.04 | KIP-371: Add a configuration to build custom SSL principal name | 2. | 2.0 (WIP) |
5 | KIP | 368: Allow SASL Connections to Periodically Re-Authenticate-497: Add inter-broker API to alter ISR | 2. | 24.0 (WIP) | |
6 | KIP-367 Introduce close(Duration) to Producer and AdminClient instead of close(long, TimeUnit)496: Administrative API to delete consumer offsets | 2.24.0 (WIP) | |||
7 | KIP-366: Make FunctionConversions deprecated495: Dynamically Adjust Log Levels in Connect | 2.14.0 (WIP) | |||
8 | KIP-365488: MaterializedClean up Sum, Serialized, Joined, Consumed and Produced with implicit SerdeCount,Total Metrics | 2.14.0 | |||
9 | KIPKIP-361: Add Consumer Configuration to Disable Auto Topic Creation484: Expose metrics for group and transaction metadata loading duration | 2.24.0 (WIP) | |||
10 | KIP-359: Verify leader epoch in produce requests481: SerDe Improvements for Connect Decimal type in JSON | 2.24.0 (WIP) | |||
11 | KIP- | 358: Migrate Streams API to Duration instead of long ms times480: Sticky Partitioner | 2. | 24.0 | (WIP)|
12 | KIP-357479: Add support to list ACLs per principalMaterialized to Join | 2.14.0(WIP) | |||
13 | KIP-356476: Add withCachingDisabled() to StoreBuilderJava AdminClient Interface | 2.14.0 | |||
14 | KIP- | 353: Improve Kafka Streams Timestamp Synchronization475: New Metrics to Measure Number of Tasks on a Connector | 2. | 14.0 (WIP) | |
15 | KIP- | 342 Add support for custom SASL extensions in OAuthBearer authentication474: To deprecate WindowStore#put(key, value) | 2. | 14.0 (WIP) | |
16 | KIP- | 341: Update Sticky Assignor's User Data Protocol471: Expose RocksDB Metrics in Kafka Streams | 2. | 24.0 (WIP) | |
17 | KIP-339: Create a new IncrementalAlterConfigs API467: Augment ProduceResponse error messaging for specific culprit records | 2.24.0 (WIP) | |||
18 | KIP-338 Support to exclude the internal topics in kafka-topics.sh command465: Add Consolidated Connector Endpoint to Connect REST API | 2.13.0 | |||
19 | KIP-340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file464: Defaults for AdminClient#createTopic | 2.14.0 | |||
20 | KIP-336: Consolidate ExtendedSerializer/Serializer and ExtendedDeserializer/Deserializer462: Use local thread id for KStreams | 2.13.0 | |||
21 | KIP- | 332: Update AclCommand to use AdminClient API461: Improve Replica Fetcher behavior at handling partition failure | 2. | 13.0 | |
22 | KIP- | 330: Add retentionPeriod in SessionBytesStoreSupplier460: Admin Leader Election RPC | 2. | 14.0 | |
23 | KIP- | 328: Ability to suppress updates for KTables458: Connector Client Config Override Policy | 2. | 23.0 | (WIP)|
24 | KIP-324: Add method to get metrics() in AdminClient455: Create an Administrative API for Replica Reassignment | 2.14.0 (WIP) | |||
25 | KIP- | 322: Return new error code for DeleteTopics API when topic deletion disabled.454: Expansion of the ConnectClusterState interface | 2. | 13.0 | |
26 | KIP-321: Update TopologyDescription to better represent Source and Sink Nodes453: Add close() method to RocksDBConfigSetter | 2.32.1.0 | |||
27 | KIP- | 320: Allow fetchers to detect and handle log truncation449: Add connector contexts to log messages in Connect workers | 2. | 13.0 | |
28 | KIP-319: Replace segments with segmentInterval in WindowBytesStoreSupplier446: Add changelog topic configuration to KTable suppress | 2.14.0 (WIP) | |||
29 | KIP-313: Add KStream.flatTransform and KStream.flatTransformValues 445: In-memory Session Store | 2.23.0 (WIP) | |||
30 | 312 Add Overloaded StreamsBuilder Build Method to Accept java.util.Properties | 2.13.0 | |||
31 | KIP- | 308: Support dynamic update of max.connections.per.ip/max.connections.per.ip.overrides442: Return to default max poll interval in Streams | 2.3.0 | ||
2.1.0 | 32 | KIP-306: Configuration for Delaying Response to Failed Authentication440: Extend Connect Converter to support headers | 2.14.0 (WIP) | ||
33 | KIP- | 305436: Add | Connect primitive number convertersa metric indicating start time | 2. | 03.0 |
34 | KIP-303434: Add Dynamic Routing in Streams SinkReplica Fetcher and Log Cleaner Count Metrics | 2.04.0 (WIP) | |||
35 | KIP-302 - Enable Kafka clients to use all DNS resolved IP addresses430 - Return Authorized Operations in Describe Responses | 2.13.0 | |||
36 | KIP-298: Error Handling in Connect429: Kafka Consumer Incremental Rebalance Protocol | 2.04.0 (WIP) | |||
37 | KIP- | 297: Externalizing Secrets for Connect Configurations428: Add in-memory window store | 2. | 03.0 | |
38 | KIP- | 295 Add Streams Configuration Allowing for Optional Topology Optimization427: Add AtMinIsr topic partition category (new metric & TopicCommand option) | 2. | 03.0 | |
39 | KIP-294 - Enable TLS hostname verification by default425: Add some Log4J Kafka Appender Properties for Producing to Secured Brokers | 2.03.0 | |||
40 | KIP-292: Add transformValues() method to KTable421: Support resolving externalized secrets in AbstractConfig | 2.03.0 | |||
41 | KIP- | 290: Support for Prefixed ACLs420: Add Single Value Fetch in Session Stores | 2. | 02.0 | |
42 | KIP- | 289: Improve the default group id behavior in KafkaConsumer417: Allow JmxTool to connect to a secured RMI port | 2. | 23.0 | (WIP)|
43 | KIP-285: Connect Rest Extension Plugin415: Incremental Cooperative Rebalancing in Kafka Connect | 2.03.0 | |||
44 | KIP-284: Set default retention ms for Streams repartition topics to Long.MAX_VALUE414: Expose Embedded ClientIds in Kafka Streams | 2.02.0 | |||
45 | KIP- | 283: Efficient Memory Usage for Down-Conversion412: Extend Admin API to support dynamic application log levels | 2. | 04.0 (WIP) | |
46 | 282: Add the listener name to the authentication context | 2. | 03.0 | ||
47 | KIP-281: ConsumerPerformance: Increase Polling Loop Timeout and Make It Reachable by the End User402: Improve fairness in SocketServer processors | 2.2.0 (partially implemented) / 2.3.0 | |||
48 | KIP- | 279: Fix log divergence between leader and follower after fast leader fail over401: TransformerSupplier/ProcessorSupplier StateStore connecting | 2. | 04.0 (WIP) | |
49 | KIP-278 - Add version option to Kafka's commands396: Add Commit/List Offsets Operations to AdminClient | 2.04.0 (WIP) | |||
50 | KIP-277 - Fine Grained ACL for CreateTopics API394: Require member.id for initial join group request | 2.02.0 | |||
51 | KIP- | 276 - Add StreamsConfig prefix for different consumers393: Time windowed serde to properly deserialize changelog input topic | 2. | 02.0 | |
52 | KIP-274: Kafka Streams Skipped Records Metrics389: Introduce a configurable consumer group size limit | 2.02.0 | |||
53 | KIP-272: Add API version tag to broker's RequestsPerSec metric386: Standardize on Min/Avg/Max metrics' default value | 2.02.0 | |||
54 | KIP-270 - A Scala Wrapper Library for Kafka Streams382: MirrorMaker 2.0 | 2.04.0 | |||
55 | 268: Simplify Kafka Streams Rebalance Metadata Upgrade | 2.04.0 (WIP) | |||
56 | KIP- | 267: Add Processor Unit Test Support to Kafka Streams Test Utils380: Detect outdated control requests and bounced brokers using broker generation | 2. | 02.0 | |
57 | KIP-266: Fix consumer indefinite blocking behavior377: TopicCommand to use AdminClient | 2.02.0 | |||
58 | KIP- | 265: Make Windowed Serde to public APIs376: Implement AutoClosable on appropriate classes that want to be used in a try-with-resource statement | 2. | 02.0 | |
59 | KIP- | 261: Add Single Value Fetch in Window Stores374: Add '--help' option to all available Kafka CLI commands | 2. | 02.0 | |
60 | KIP- | 257 - Configurable Quota Management372: Naming Repartition Topics for Joins and Grouping | 2. | 01.0 | |
61 | KIP- | 255: OAuth Authentication via SASL/OAUTHBEARER371: Add a configuration to build custom SSL principal name | 2. | 02.0 | |
62 | KIP-251: Allow timestamp manipulation in Processor API369: Alternative Partitioner to Support "Always Round-Robin" Selection | 2.04.0 | |||
63 | KIP -249: Add Delegation Token Operations to KafkaAdminClient368: Allow SASL Connections to Periodically Re-Authenticate | 2.02.0 | |||
64 | KIP- | 247: Add public test utils for Kafka Streams367 Introduce close(Duration) to Producer and AdminClient instead of close(long, TimeUnit) | 2.2 | 1.1.0 | |
65 | KIP-245: Use Properties instead of StreamsConfig in KafkaStreams constructor366: Make FunctionConversions deprecated | 2.01.0 | |||
66 | KIP-244: Add Record Header support to Kafka Streams Processor API365: Materialized, Serialized, Joined, Consumed and Produced with implicit Serde | 2.01.0 | |||
67 | KIPKIP- | 243: Make ProducerConfig and ConsumerConfig constructors public361: Add Consumer Configuration to Disable Auto Topic Creation | 2.3 | 1.1.0 | |
68 | KIP- | 239 Add queryableStoreName() to GlobalKTable360: Improve handling of unknown producer | 2.4.0 (WIP) | 1.1.0||
69 | KIP- | 238: Expose Kafka cluster ID in Connect REST API359: Verify leader epoch in produce requests | 2.4.0 (WIP) | 1.1.0||
70 | KIP- | 237: More Controller Health Metrics358: Migrate Streams API to Duration instead of long ms times | 2. | 01.0 | |
71 | KIP- | 235357: Add | DNS alias support for secured connectionsupport to list ACLs per principal | 2.1.0 | |
72 | KIP- | 233: Simplify StreamsBuilder#addGlobalStore356: Add withCachingDisabled() to StoreBuilder | 2 | 1.1.0 | |
73 | KIP-231: Improve the Required ACL of ListGroups API354: Add a Maximum Log Compaction Lag | 2.13.0 | |||
74 | KIP- | 229: DeleteGroups API353: Improve Kafka Streams Timestamp Synchronization | 2 | 1.1.0 | |
75 | KIP- | 227 - Introduce Incremental FetchRequests to Increase Partition Scalability352: Distinguish URPs caused by reassignment | 2.4.0 (WIP) | 1.1.0||
76 | KIP | -226 - Dynamic Broker Configuration-351: Add --under-min-isr option to describe topics command | 2.3 | 1.1.0 | |
77 | KIP- | 225 - Use tags for consumer “records.lag” metrics346: Improve LogCleaner behavior on error | 2.1 | 1.1.0||
78 | KIP- | 224: Add configuration parameter `retries` to Streams API1.1.0 | |||
79 | KIP-223 - Add per-topic min lead and per-partition lead metrics to KafkaConsumer | 2.0.0 | |||
80 | KIP-222 - Add Consumer Group operations to Admin API | 2.0.0 | |||
81 | KIP-220: Add AdminClient into Kafka Streams' ClientSupplier | 1.1.0 | |||
82 | KIP-219 - Improve quota communication | 2.0.0 | |||
345: Introduce static membership protocol to reduce consumer rebalances | 2.4.0 (WIP), partially available in 2.3.0 | ||||
79 | KIP-342 Add support for custom SASL extensions in OAuthBearer authentication | 2.1.0 | |||
80 | KIP-341: Update Sticky Assignor's User Data Protocol | 2.3.0 | |||
81 | KIP-340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file | 2.1.0 | |||
82 | KIP-339: Create a new IncrementalAlterConfigs API | 2.3.0 | |||
83 | KIP-338 Support to exclude the internal topics in kafka-topics.sh command | 2 | 83 | KIP-218: Make KafkaFuture.Function java 8 lambda compatible | 1.1.0 |
84 | KIP- | 215: Add topic regex support for Connect sinks336: Consolidate ExtendedSerializer/Serializer and ExtendedDeserializer/Deserializer | 2 | 1.1.0 | |
85 | KIP-214: Add zookeeper.max.in.flight.requests config to the broker332: Update AclCommand to use AdminClient API | 21.1.0 | |||
86 | KIP- | 212: Enforce set of legal characters for connector names331 Add default implementation to close() and configure() for Serializer, Deserializer and Serde | 2.3 | 1.1.0 | |
87 | KIP- | 211: Revise Expiration Semantics of Consumer Group Offsets330: Add retentionPeriod in SessionBytesStoreSupplier | 2.1.0 | ||
88 | KIP- | 210 - Provide for custom error handling when Kafka Streams fails to produce328: Ability to suppress updates for KTables | 2.1.0 (partially implemented) / 2.3.0 (WIP) | 1.1.0||
89 | KIP- | 208324: Add | SSL support to Kafka Connect REST interfacemethod to get metrics() in AdminClient | 2 | 1.1.0 |
90 | KIP-207: Offsets returned by ListOffsetsResponse should be monotonically increasing even during a partition leader change322: Return new error code for DeleteTopics API when topic deletion disabled. | 2.21.0 (WIP) | |||
91 | KIP-206: Add support for UUID serialization and deserialization321: Update TopologyDescription to better represent Source and Sink Nodes | 2.1.0 | |||
92 | KIP- | 205: Add all() and range() API to ReadOnlyWindowStore320: Allow fetchers to detect and handle log truncation | 2.1.0 (partially implemented) / 2.2 | 1.1.0 | |
93 | KIP- | 204 : Adding records deletion operation to the new Admin Client API319: Replace segments with segmentInterval in WindowBytesStoreSupplier | 2 | 1.1.0 | |
94 | KIP- | 203313: Add | toLowerCase support to sasl.kerberos.principal.to.local ruleKStream.flatTransform and KStream.flatTransformValues | 2.2.0 (partially implemented) / 2.3 | 1.1.0 |
95 | KIP- | 202: Move merge() from StreamsBuilder to KStream312 Add Overloaded StreamsBuilder Build Method to Accept java.util.Properties | 2.1 | 1.0.0 | |
96 | KIP-198: Remove ZK dependency from Streams Reset Tool308: Support dynamic update of max.connections.per.ip/max.connections.per.ip.overrides | 2.11.0.0 | |||
97 | KIP- | 197 Connect REST API should include the connector type when describing a connector307: Allow to define custom processor names with KStreams DSL | 2.3.0 (partial) | 1.0.0||
98 | KIP- | 196: Add metrics to Kafka Connect framework306: Configuration for Delaying Response to Failed Authentication | 2.1 | 1.0.0 | |
99 | KIP-195: AdminClient.createPartitions305: Add Connect primitive number converters | 21.0.0 | |||
100 | KIP- | 192 : Provide cleaner semantics when idempotence is enabled303: Add Dynamic Routing in Streams Sink | 2 | 1.0.0 | |
101 | KIP | -191: KafkaConsumer.subscribe() overload that takes just Pattern-302 - Enable Kafka clients to use all DNS resolved IP addresses | 2.1 | 1.0.0 | |
102 | KIP- | 190: Handle client-ids consistently between clients and brokers300: Add Windowed KTable API in StreamsBuilder | 2.4 | 1.0.0 | |
103 | KIP- | 189: Improve principal builder interface and add support for SASL298: Error Handling in Connect | 2 | 1.0.0 | |
104 | KIP- | 188 - Add new metrics to support health checks297: Externalizing Secrets for Connect Configurations | 2 | 1.0.0 | |
105 | KIP- | 187 - Add cumulative count metric for all Kafka rate metrics295 Add Streams Configuration Allowing for Optional Topology Optimization | 2 | 1.0.0 | |
106 | KIP | -186: Increase offsets retention default to 7 days-294 - Enable TLS hostname verification by default | 2.0.0 | ||
107 | 183 - Change PreferredReplicaLeaderElectionCommand to use AdminClient | Accepted | |||
108 | KIP-292: Add transformValues() method to KTable | 2.0.0 | |||
108109 | KIP- | 182: Reduce Streams DSL overloads and allow easier use of custom storage engines290: Support for Prefixed ACLs | 2 | 1.0.0 | |
109110 | KIP- | 180: Add a broker metric specifying the number of consumer group rebalances in progress289: Improve the default group id behavior in KafkaConsumer | 2.2 | 1.1.0 | |
110111 | KIP-177: Consumer perf tool should count rebalance time285: Connect Rest Extension Plugin | 21.0.0 | |||
111112 | KIP-176: Remove deprecated new-consumer option for tools284: Set default retention ms for Streams repartition topics to Long.MAX_VALUE | 2.0.0 | |||
112113 | KIP- | 175: Additional '--describe' views for ConsumerGroupCommand1.1.0 | 113 | KIP-174 - Deprecate and remove internal converter configs in WorkerConfig283: Efficient Memory Usage for Down-Conversion | 2.0.0 |
114 | KIP- | 173282: Add | prefix to StreamsConfig to enable setting default internal topic configsthe listener name to the authentication context | 2 | 1.0.0 |
115 | KIP- | 171 - Extend Consumer Group Reset Offset for Stream Application281: ConsumerPerformance: Increase Polling Loop Timeout and Make It Reachable by the End User | 2.0 | 1.1.0 | |
116 | KIP-168: Add GlobalTopicCount and GlobalPartitionCount metric per cluster279: Fix log divergence between leader and follower after fast leader fail over | 21.0.0 | |||
117 | KIP-278 - | 167: Add interface for the state store restoration processAdd version option to Kafka's commands | 2 | 1.0.0 | |
118 | KIP- | 164 - Add UnderMinIsrPartitionCount and per-partition UnderMinIsr metrics277 - Fine Grained ACL for CreateTopics API | 2 | 1.0.0 | |
119 | KIP- | 163: Lower the Minimum Required ACL Permission of OffsetFetch276 - Add StreamsConfig prefix for different consumers | 2 | 1.0.0 | |
120 | KIP-162: Enable topic deletion by default274: Kafka Streams Skipped Records Metrics | 21.0.0 | |||
121 | KIP- | 161: streams deserialization exception handlers272: Add API version tag to broker's RequestsPerSec metric | 2 | 1.0.0 | |
122 | KIP | -160: Augment KStream.print(), KStream.writeAsText() to allow users pass in extra parameters in the printed string-270 - A Scala Wrapper Library for Kafka Streams | 2 | 1.0.0 | |
123 | KIP- | 157 - Add consumer config options to streams reset tool268: Simplify Kafka Streams Rebalance Metadata Upgrade | 2 | 1.0.0 | |
124 | KIP- | 156 Add option "dry run" to Streams application reset tool267: Add Processor Unit Test Support to Kafka Streams Test Utils | 2 | 0.11.0.0 | |
125 | KIP-155 - Add range scan for windowed state stores266: Fix consumer indefinite blocking behavior | 20.11.0.0 | |||
126 | KIP-154 Add Kafka Connect configuration properties for creating internal topics265: Make Windowed Serde to public APIs | 2.0.00.11.0.0 | |||
127 | KIP-153: Include only client traffic in BytesOutPerSec metric261: Add Single Value Fetch in Window Stores | 20.11.0.0 | |||
128 | KIP- | 152 - Improve diagnostics for SASL authentication failures258: Allow to Store Record Timestamps in RocksDB | 2.3.0 (partially implemented) | 1.0.0||
129 | KIP-151 Expose Connector type in REST API-257 - Configurable Quota Management | 20.11.0.0 | |||
130 | KIP- | 150 - Kafka-Streams Cogroup255: OAuth Authentication via SASL/OAUTHBEARER | 2. | 2.0 (WIP)0.0 | |
131 | KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner | 1.1.0 (partially implemented) (WIP for 2.2.0) | 251: Allow timestamp manipulation in Processor API | 2.0.0 | |
132 | KIP-249: Add Delegation Token Operations to KafkaAdminClient | 2. | 132 | KIP-146 - Classloading Isolation in Connect | 0.11.0.0 |
133 | KIP-145 - Expose Record Headers in Kafka Connect247: Add public test utils for Kafka Streams | 1.1.0 | |||
134 | KIP-144: Exponential backoff for broker reconnect attempts245: Use Properties instead of StreamsConfig in KafkaStreams constructor | 2.0.11.0.0 | |||
135 | KIP- | 143: Controller Health Metrics244: Add Record Header support to Kafka Streams Processor API | 2 | 0.11.0.0 | |
136 | KIP-140: Add administrative RPCs for adding, deleting, and listing ACLs243: Make ProducerConfig and ConsumerConfig constructors public | 1.1.00.11.0.0 | |||
137 | KIP-138: Change punctuate semantics239 Add queryableStoreName() to GlobalKTable | 1.01.0 | |||
138 | KIP-137: Enhance TopicCommand --describe to show topics marked for deletion238: Expose Kafka cluster ID in Connect REST API | 1.10.11.0.0 | |||
139 | KIP-136: Add Listener name to SelectorMetrics tags237: More Controller Health Metrics | 2.0.11.0.0 | |||
140 | KIP- | 134: Delay initial consumer group rebalance235: Add DNS alias support for secured connection | 2.1 | 0.11.0.0 | |
141 | KIP-133: Describe and Alter Configs Admin APIs233: Simplify StreamsBuilder#addGlobalStore | 1.1.00.11.0.0 | |||
142 | KIP-130: Expose states of active tasks to KafkaStreams public API231: Improve the Required ACL of ListGroups API | 2.11.0.0 | |||
143 | KIP-129: Streams Exactly-Once Semantics229: DeleteGroups API | 1.10.11.0.0 | |||
144 | KIP-128: Add ByteArrayConverter for Kafka Connect-227 - Introduce Incremental FetchRequests to Increase Partition Scalability | 1.10.11.0.0 | |||
145 | KIP-126 - Allow KafkaProducer to split and resend oversized batches.226 - Dynamic Broker Configuration | 1.10.11.0.0 | |||
146 | KIP-124 - Request rate quotas225 - Use tags for consumer “records.lag” metrics | 1.10.11.0.0 | |||
147 | KIP-123: Allow per stream/table timestamp extractor224: Add configuration parameter `retries` to Streams API | 1.10.11.0.0 | |||
148 | KIP-223 - 122: Add Reset Consumer Group Offsets toolingAdd per-topic min lead and per-partition lead metrics to KafkaConsumer | 2.0.11.0.0 | |||
149 | KIP-121: Add KStream peek method222 - Add Consumer Group operations to Admin API | 2.0.11.0.0 | |||
150 | KIP-120: Cleanup Kafka Streams builder API221: Enhance DSL with Connecting Topic Creation and Repartition Hint | 2.4.0 (WIP)1.0.0 | |||
151 | KIP-119: Drop Support for Scala 2.10 in Kafka 0.220: Add AdminClient into Kafka Streams' ClientSupplier | 1.1110.11.0.0 | |||
152 | KIP-118: Drop Support for Java 7-219 - Improve quota communication | 2.0.0 | |||
153 | KIP-117: Add a public AdminClient API for Kafka admin operations218: Make KafkaFuture.Function java 8 lambda compatible | 1.10.11.0.0 | |||
154 | KIP-115215: Enforce offsets.topic.replication.factor upon __consumer_offsets auto topic creationAdd topic regex support for Connect sinks | 1.10.11.0.0 | |||
155 | KIP-114: KTable state stores and improved semantics214: Add zookeeper.max.in.flight.requests config to the broker | 1.10.11.0.0 | |||
156 | KIP- | 113: Support replicas movement between log directories213 Support non-key joining in KTable | 2.4.0 (WIP) | 1.1.0||
157 | KIP-112: Handle disk failure for JBOD212: Enforce set of legal characters for connector names | 1.01.0 | |||
158 | KIP- | 110: Add Codec for ZStandard Compression211: Revise Expiration Semantics of Consumer Group Offsets | 2.1.0 | ||
159 | KIP-109: Old Consumer Deprecation-210 - Provide for custom error handling when Kafka Streams fails to produce | 1.10.11.0.0 | |||
160 | KIP-108: Create Topic Policy208: Add SSL support to Kafka Connect REST interface | 1.1.0.10.2.0 | |||
161 | KIP-107: Add deleteRecordsBefore() API in AdminClient | 0.11.0.0 | 207: Offsets returned by ListOffsetsResponse should be monotonically increasing even during a partition leader change | 2.2.0 | |
162 | KIP-206: Add support for UUID serialization and deserialization | 2.1 | 162 | KIP-106 - Change Default unclean.leader.election.enabled from True to False | 0.11.0.0 |
163 | KIP-105: Addition of Record Level for Sensors205: Add all() and range() API to ReadOnlyWindowStore | 1.1.0.10.2.0 | |||
164 | KIP-104: Granular Sensors for Streams 204 : Adding records deletion operation to the new Admin Client API | 1.10.10.2.0 | |||
165 | KIP-103: Separation of Internal and External traffic203: Add toLowerCase support to sasl.kerberos.principal.to.local rule | 1.10.10.2.0 | |||
166 | KIP-102 - Add close with timeout for consumers202: Move merge() from StreamsBuilder to KStream | 1.0.0.10.2.0 | |||
167 | KIP-101 - Alter Replication Protocol to use Leader Epoch rather than High Watermark for Truncation198: Remove ZK dependency from Streams Reset Tool | 1.0.11.0.0 | |||
168 | KIP-100 - Relax Type constraints in Kafka Streams API197 Connect REST API should include the connector type when describing a connector | 1.0.0.10.2.0 | |||
169 | KIP-99196: Add Global Tables metrics to Kafka StreamsConnect framework | 1.0.10.2.0 | |||
170 | KIP-98 - Exactly Once Delivery and Transactional Messaging195: AdminClient.createPartitions | 10.11.0.0 | |||
171 | KIP-97: Improved Kafka Client RPC Compatibility Policy192 : Provide cleaner semantics when idempotence is enabled | 1.00.10.2.0 | |||
172 | KIP-96 - Add per partition metrics for in-sync and assigned replica count191: KafkaConsumer.subscribe() overload that takes just Pattern | 1.00.10.2.0 | |||
173 | KIP-94 Session Windows-190: Handle client-ids consistently between clients and brokers | 1.00.10.2.0 | |||
174 | KIP-93189: Improve invalid timestamp handling in Kafka Streamsprincipal builder interface and add support for SASL | 1.0.0.10.2.0 | |||
175 | KIP-92 188 - Add per partition lag new metrics to KafkaConsumersupport health checks | 1.0.0 | |||
176 | KIP-187 - Add cumulative count metric for all Kafka rate metrics | 1.010.2.0 | |||
176177 | KIP-91 Provide Intuitive User Timeouts in The Producer186: Increase offsets retention default to 7 days | 2.10.0 | |||
177178 | KIP-90 - Remove zkClient dependency from Streams183 - Change PreferredReplicaLeaderElectionCommand to use AdminClient | 20.10.2.0 | |||
178179 | KIP-89: Allow sink connectors to decouple flush and offset commit | 0.10.2.0 | |||
179 | KIP-88: OffsetFetch Protocol Update | 0.10.2.0 | |||
182: Reduce Streams DSL overloads and allow easier use of custom storage engines | 1.0.0 | ||||
180 | KIP-180: Add a broker metric specifying the number of consumer group rebalances in progress | 1.1.0 | |||
181 | KIP-177: Consumer perf tool should count rebalance time | 1 | 180 | KIP-86: Configurable SASL callback handlers | 2.0.0 |
181182 | KIP- | 85: Dynamic JAAS configuration for Kafka clients176: Remove deprecated new-consumer option for tools | 2.0 | 0.10.2.0 | |
182183 | KIP-84: Support SASL SCRAM mechanisms | 0.10.2.0 | |||
183 | KIP-82 - Add Record Headers | 0.11.0.0 | |||
184 | KIP-81: Bound Fetch memory usage in the consumer | 2.2.0 (WIP) | |||
175: Additional '--describe' views for ConsumerGroupCommand | 1.1.0 | ||||
184 | KIP-174 - Deprecate and remove internal converter configs in WorkerConfig | 2.0.0 | |||
185 | KIP-173: Add prefix to StreamsConfig to enable setting default internal topic configs | 1.0. | 185 | KIP-79 - ListOffsetRequest/ListOffsetResponse v1 and add timestamp search methods to the new consumer | 0.10.1.0 |
186 | KIP-78: Cluster Id-171 - Extend Consumer Group Reset Offset for Stream Application | 10.10.1.0 | |||
187 | KIP-77: Improve Kafka Streams Join Semantics168: Add GlobalTopicCount and GlobalPartitionCount metric per cluster | 1.00.10.2.0 | |||
188 | KIP-75 - Add per-connector Converters167: Add interface for the state store restoration process | 1.0.0.10.1.0 | |||
189 | 74: Add Fetch Response Size Limit in Bytes Add UnderMinIsrPartitionCount and per-partition UnderMinIsr metrics | 1.00.10.1.0 | |||
190 | 73: Replication Quotas | 1.0. | 0.10.1.0 | ||
191 | KIP-72: Allow putting a bound on memory consumed by Incoming request 162: Enable topic deletion by default | 1.0.0 | |||
192 | KIP-71: Enable log compaction and deletion to co-exist161: streams deserialization exception handlers | 1.00.10.1.0 | |||
193 | KIP-70: Revise Partition Assignment Semantics on New Consumer's Subscription Change160: Augment KStream.print(), KStream.writeAsText() to allow users pass in extra parameters in the printed string | 1.0.00.10.1.0 | |||
194 | KIP | -67: Queryable state for Kafka Streams-157 - Add consumer config options to streams reset tool | 1.0 | 0.10.1.0 | |
195 | KIP- | 66: Single Message Transforms for Kafka Connect156 Add option "dry run" to Streams application reset tool | 0. | 1011. | 20.0 |
196 | KIP-155 - Add range scan for windowed state stores | /0.11.0.0 | |||
196197 | KIP-65: Expose timestamps to Connect154 Add Kafka Connect configuration properties for creating internal topics | 0.1011.10.0 | |||
197198 | KIP-63: Unify store and downstream caching in streams153: Include only client traffic in BytesOutPerSec metric | 0.10.1.0 | 198 | KIP-62: Allow consumer to send heartbeats from a background thread | 0.10.1.11.0.0 |
199 | KIP-60 - Make Java client classloading more flexible152 - Improve diagnostics for SASL authentication failures | 1.00.10.1.0 | |||
200 | KIP-58 - Make Log Compaction Point Configurable151 Expose Connector type in REST API | 0.1011.10.0 | |||
201 | KIP-150 - 57 - Interoperable LZ4 FramingKafka-Streams Cogroup | 2.4.0 (WIP)0.10.0.0 | |||
202 | KIP-56: Allow cross origin HTTP requests on all HTTP methods149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner | 1.1.0 (partially implemented) (WIP for 2.4.0)0.10.0.0 | |||
203 | KIP-55: Secure Quotas for Authenticated Users146 - Classloading Isolation in Connect | 0. 1011. 10.0 | |||
204 | KIP-54: Sticky Partition Assignment Strategy145 - Expose Record Headers in Kafka Connect | 1.10.11.0.0 | |||
205 | KIP-52: Connector Control APIs144: Exponential backoff for broker reconnect attempts | 0.1011.0.0 | |||
206 | KIP-51 - List Connectors REST API143: Controller Health Metrics | 0.1011.0.0 | |||
207 | KIP-50 - Move Authorizer to o.a.k.common package140: Add administrative RPCs for adding, deleting, and listing ACLs | 0.1011.10.0 | |||
208 | KIP-48 Delegation token support for Kafka138: Change punctuate semantics | 1.10.0 | |||
209 | KIP-45 - Standardize all client sequence interaction on j.u.Collection.137: Enhance TopicCommand --describe to show topics marked for deletion | 0.1011.0.0 | |||
210 | KIP-43: Kafka SASL enhancements136: Add Listener name to SelectorMetrics tags | 0.1011.0.0 | |||
211 | KIP-42: Add Producer and Consumer Interceptors134: Delay initial consumer group rebalance | 0.1011.0.0 | |||
212 | KIP-41: Consumer Max Records133: Describe and Alter Configs Admin APIs | 0.1011.0.0 | |||
213 | KIP- | 40: ListGroups and DescribeGroup130: Expose states of active tasks to KafkaStreams public API | 1 | 0.9.0.0 | |
214 | KIP-38: ZooKeeper Authentication129: Streams Exactly-Once Semantics | 0.911.0.0 | |||
215 | KIP-36 - Rack aware replica assignment128: Add ByteArrayConverter for Kafka Connect | 0.1011.0.0 | |||
216 | KIP-35 - Retrieving protocol version126 - Allow KafkaProducer to split and resend oversized batches. | 0.1011.0.0 | |||
217 | KIP-33 - Add a time based log index124 - Request rate quotas | 0.1011.10.0 | |||
218 | KIP-32 - Add timestamps to Kafka message123: Allow per stream/table timestamp extractor | 0.1011.0.0 | |||
219 | KIP-31 - Move to relative offsets in compressed message sets122: Add Reset Consumer Group Offsets tooling | 0.1011.0.0 | |||
220 | KIP-28 - Add a processor client121: Add KStream peek method | 0. 1011.0.0 | |||
221 | KIP-26 - Add Kafka Connect framework for data import/export120: Cleanup Kafka Streams builder API | 10.9.0.0 | |||
222 | KIP-25 - System test improvements119: Drop Support for Scala 2.10 in Kafka 0.11 | 0.911.0.0 | |||
223 | KIP-22 - Expose a Partitioner interface in the new producer118: Drop Support for Java 7 | 20.9.0.0 | |||
224 | KIP-21 - Dynamic Configuration117: Add a public AdminClient API for Kafka admin operations | 0.911.0.0 (WIP) | |||
225 | KIP-20 Enable log preallocate to improve consume performance under windows and some old Linux file system115: Enforce offsets.topic.replication.factor upon __consumer_offsets auto topic creation | 0.110.9.0.0 | |||
226 | KIP-19 - Add a request timeout to NetworkClient114: KTable state stores and improved semantics | 0.911.0.0 | |||
227 | KIP-16 - Automated Replica Lag Tuning113: Support replicas movement between log directories | 1.10.9.0.0 | |||
228 | KIP-15 - Add a close method with a timeout in the producer112: Handle disk failure for JBOD | 1.0.9.0.0 | |||
229 | KIP- | 13 - Quota Design110: Add Codec for ZStandard Compression | 2.1.0 | 0.9.0.0||
230 | KIP-12 - Kafka Sasl/Kerberos and SSL implementation109: Old Consumer Deprecation | 0.911.0.0 | |||
231 | KIP-11 - Kafka Authorizer design108: Create Topic Policy | 0.910.02.0 | |||
232 | KIP-8 - Add a flush method to the producer API107: Add deleteRecordsBefore() API in AdminClient | 0.911.0.0 | |||
233 | 4 - Metadata Protocol Changes 106 - Change Default unclean.leader.election.enabled from True to False | 0.1011.0.0 | |||
234 | KIP-4 - Command line and centralized administrative operations105: Addition of Record Level for Sensors | 0.910.02.0, | |||
235 | KIP-104: Granular Sensors for Streams | 0.10.02.0, | |||
236 | KIP-103: Separation of Internal and External traffic | 0.10.12.0 | |||
235237 | KIP-3 - Mirror Maker Enhancement102 - Add close with timeout for consumers | 0.910.02.0 | |||
236238 | KIP-2 - Refactor brokers to allow listening on multiple ports and IPs101 - Alter Replication Protocol to use Leader Epoch rather than High Watermark for Truncation | 0.911.0.0 | |||
237239 | KIP-1 - Remove support of request.required.acks100 - Relax Type constraints in Kafka Streams API | 0.910.02.0 |
...
...
KIPs under discussion
Please insert new rows in sorted order (ascending by KIP number). Please move the adopted/accepted KIPs to the "Adopted KIPs" table above.
Dormant/inactive KIPs
Please insert new rows in sorted order (ascending by KIP number).
Discarded KIPs
Please insert new rows in sorted order (ascending by KIP number).
...
Discarded KIPs
KIP | Comment |
---|---|
KIP-5 - Broker Configuration Management | Superseded by KIP-21 |
KIP-7 - Security - IP Filtering | |
KIP-9 - SSL Support | Adopted via KAFKA-1690 but not via this KIP process |
KIP-18 - JBOD Support | Superseded by KIP-112 and KIP-113 |
KIP-24 - Remove ISR information from TopicMetadataRequest and add broker level metadata request | |
KIP-29 - Add an IsrPropagateIntervalMs configuration to KafkaConfig | No longer need after KAFKA-2722 |
KIP-34 Add Partitioner Change Listener to Partitioner Interface for Multiple Use Case | Per feedback it is better do this parition() method and avoid Thread Coordination etc. |
KIP-80: Kafka Rest Server | |
KIP-83 - Allow multiple SASL authenticated Java clients in a single JVM process | After KIP-85, no interface changes, reduced to KAFKA-4180 |
KIP-116: Add State Store Checkpoint Interval Configuration | Currently not needed as checkpointing can be done on commit interval. |
KIP-127: Pluggable JAAS LoginModule configuration for SSL | Similar functionality can be implemented using a custom PrincipalBuilder. |
KIP-132: Augment KStream.print to allow extra parameters in the printed string | Duplicated by KIP-160 |
KIP-184: Rename LogCleaner and related classes to LogCompactor | A lot of configuration change will cause more trouble and doesn't seems to be worth it |
KIP-111: Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker. | Covered by KIP-189 |
KIP-147: Add missing type parameters to StateStoreSupplier factories and KGroupedStream/Table methods | Covered by KIP-182 |
KIP-170: Enhanced TopicCreatePolicy and introduction of TopicDeletePolicy | Superseded by KIP-201: Rationalising Policy interfaces |
KIP 172: Add regular-expression topic support for sink connector | Superseded by KIP-215: Add topic regex support for Connect sinks | KIP-179: Change ReassignPartitionsCommand to use AdminClient |
KIP 230: Name Windowing Joins | Covered by KIP-372: Naming Repartition Topics for Joins and Grouping |
KIP-232: Detect outdated metadata using per-partition leaderEpoch field | Merged into KIP-320: Allow fetchers to detect and handle log truncation |
KIP-246: Connect producers and consumers should inherit worker configs | Withdrawn because proposal could not be made backward compatible with existing behavior |
KIP-263: Allow broker to skip sanity check of inactive segments on broker startup | Withdrawn because the solution we agreed on does not require interface change |
KIP-286: producer.send() should not block on metadata update | Withdrawn because the benefit of not having to wait for metadata is probably not worth the complexity added in producer. |
KIP-288: [DISCARDED] Consumer.poll() timeout semantic change and new waitForAssignment method | Discarded in deference to KIP-266 |
KIP-310: Add a Kafka Source Connector to Kafka Connect | Withdrawn in favor of KIP-382. |
KIP-327: Add describe all topics API to AdminClient | Withdrawn in-favor of filter support in Metadata API and KIP-142 |
KIP-344: The auto-generated client id should be passed to MetricsReporter | Bug fix so KIP not needed |
KIP-364: Remove implicit Materialized, Consumed and Produced | Subsumed by KIP-365 |
KIP Discussion Recordings
...