...
- Click
. Take the next available KIP number and give your proposal a descriptive heading. e.g. "KIP 42: Allow Infinite Retention With Bounded Disk Usage".Create from template templateName 54329345 templateId 54329345 title KIP-NEXT: Insert Title Here buttonLabel Create KIP - Fill in the sections as described above
- Start a [DISCUSS] thread on the Apache mailing list. Please ensure that the subject of the thread is of the format [DISCUSS] KIP-{your KIP number} {your KIP heading} The and the body contains a link to your new KIP. The discussion should happen on the mailing list, not on the wiki, since the wiki comment system doesn't work well for larger discussions. In the process of the discussion you may update the proposal. You should let people know the changes you are making. When you feel you have a finalized proposal
- Once the proposal is finalized call a [VOTE] to have the proposal adopted. These proposals are more serious than code changes and more serious even than release votes. The criteria for acceptance is lazy majority. The vote should remain open for at least 72 hours.
- Please update the KIP wiki page, and the index below, to reflect the current stage of the KIP after a vote. This acts as the permanent record indicating the result of the KIP (e.g., Accepted or Rejected). Also report the result of the KIP vote to the voting thread on the mailing list so the conclusion is clear.
KIP round-up
Next KIP Number: 441529
Use this number as the identifier for your KIP and increment this value.
Adopted KIPs
Please insert new rows in sorted order (descending by KIP number).
KIP (please keep this sorted by KIP number) | Release | |||||
---|---|---|---|---|---|---|
1 | KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum | |||||
2 | KIP-482: The Kafka Protocol should Support Optional Tagged Fields | 2.4 | 427: Add AtMinIsr topic partition category (new metric & TopicCommand option)2.3.0 (WIP) | 2 | KIP-430 - Return Authorized Operations in Describe Responses | 2.3.0 (WIP) |
3 | KIP-504 - | 428: Add in-memory window storeAdd new Java Authorizer Interface | 2. | 34.0 (WIP) | ||
4 | KIP-425503: Add some Log4J Kafka Appender Properties for Producing to Secured Brokersmetric for number of topics marked for deletion | 2.34.0 (WIP) | ||||
5 | KIP-420497: Add Single Value Fetch in Session Storesinter-broker API to alter ISR | 2.24.0 (WIP) | ||||
6 | KIP-412496: Extend Admin Administrative API to support dynamic application log levelsdelete consumer offsets | 2.34.0 (WIP) | ||||
7 | KIP- | 414: Expose Embedded ClientIds in Kafka Streams495: Dynamically Adjust Log Levels in Connect | 2. | 24.0 (WIP) | ||
8 | KIP-402: Improve fairness in SocketServer processors488: Clean up Sum,Count,Total Metrics | 2.24.0 (WIP) | ||||
9 | KIP-394: Require member.id for initial join group request484: Expose metrics for group and transaction metadata loading duration | 2.24.0 (WIP) | ||||
10 | KIP-393: Time windowed serde to properly deserialize changelog input topic481: SerDe Improvements for Connect Decimal type in JSON | 2.24.0 (WIP) | ||||
11 | KIP- | 386: Standardize on Min/Avg/Max metrics' default value480: Sticky Partitioner | 2. | 24.0 | ||
12 | KIP- | 382: MirrorMaker 2.0479: Add Materialized to Join | 2. | 34.0(WIP) | ||
13 | KIP-380: Detect outdated control requests and bounced brokers using broker generation476: Add Java AdminClient Interface | 2.24.0 | ||||
14 | KIP- | 377475: | TopicCommand to use AdminClientNew Metrics to Measure Number of Tasks on a Connector | 2. | 24.0 (WIP) | |
15 | KIP- | 376: Implement AutoClosable on appropriate classes that want to be used in a try-with-resource statement474: To deprecate WindowStore#put(key, value) | 2. | 24.0 (WIP) | ||
16 | KIP- | 374: Add '--help' option to all available Kafka CLI commands471: Expose RocksDB Metrics in Kafka Streams | 2. | 24.0 (WIP) | ||
17 | KIP-372: Naming Repartition Topics for Joins and Grouping467: Augment ProduceResponse error messaging for specific culprit records | 2.14.0 (WIP) | ||||
18 | KIP- | 371465: Add | a configuration to build custom SSL principal nameConsolidated Connector Endpoint to Connect REST API | 2. | 23.0 | |
19 | KIP | 368: Allow SASL Connections to Periodically Re-Authenticate-464: Defaults for AdminClient#createTopic | 2. | 24.0 | ||
20 | KIP-367 Introduce close(Duration) to Producer and AdminClient instead of close(long, TimeUnit)462: Use local thread id for KStreams | 2.23.0 | ||||
21 | KIP- | 366: Make FunctionConversions deprecated461: Improve Replica Fetcher behavior at handling partition failure | 2. | 13.0 | ||
22 | KIP- | 365: Materialized, Serialized, Joined, Consumed and Produced with implicit Serde460: Admin Leader Election RPC | 2. | 14.0 | ||
23 | KIPKIP- | 361: Add Consumer Configuration to Disable Auto Topic Creation458: Connector Client Config Override Policy | 2.3.0 | (WIP)|||
24 | KIP-359: Verify leader epoch in produce requests455: Create an Administrative API for Replica Reassignment | 2.34.0 (WIP) | ||||
25 | KIP-358: Migrate Streams API to Duration instead of long ms times454: Expansion of the ConnectClusterState interface | 2.13.0 | ||||
26 | KIP- | 357453: Add | support to list ACLs per principalclose() method to RocksDBConfigSetter | 2. | 13.0 | |
27 | KIP- | 356449: Add | withCachingDisabled() to StoreBuilderconnector contexts to log messages in Connect workers | 2. | 13.0 | |
28 | KIP-353: Improve Kafka Streams Timestamp Synchronization446: Add changelog topic configuration to KTable suppress | 2.14.0 (WIP) | ||||
29 | KIP-351445: Add --under-min-isr option to describe topics commandIn-memory Session Store | 2.3.0 | ||||
30 | 346: Improve LogCleaner behavior on error | 2.3.02.1 | ||||
31 | KIP-342 Add support for custom SASL extensions in OAuthBearer authentication442: Return to default max poll interval in Streams | 2.13.0 | ||||
32 | KIP-341: Update Sticky Assignor's User Data Protocol440: Extend Connect Converter to support headers | 2.34.0 (WIP) | ||||
33 | KIP-340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file436: Add a metric indicating start time | 2.13.0 | ||||
34 | KIP-339: Create a new IncrementalAlterConfigs API434: Add Replica Fetcher and Log Cleaner Count Metrics | 2.34.0 (WIP) | ||||
35 | KIP-338 Support to exclude the internal topics in kafka-topics.sh command430 - Return Authorized Operations in Describe Responses | 2.13.0 | ||||
36 | KIP-336: Consolidate ExtendedSerializer/Serializer and ExtendedDeserializer/Deserializer429: Kafka Consumer Incremental Rebalance Protocol | 2.14.0 (WIP) | ||||
37 | KIP-332: Update AclCommand to use AdminClient API428: Add in-memory window store | 2.13.0 | ||||
38 | KIP- | 331 Add default implementation to close() and configure() for Serializer, Deserializer and Serde427: Add AtMinIsr topic partition category (new metric & TopicCommand option) | 2.3.0 | 2.3.0 (WIP)|||
39 | KIP-330425: Add retentionPeriod in SessionBytesStoreSuppliersome Log4J Kafka Appender Properties for Producing to Secured Brokers | 2.13.0 | ||||
40 | KIP-328: Ability to suppress updates for KTables421: Support resolving externalized secrets in AbstractConfig | 2.1.0 (partially implemented) / 2.3.0 (WIP) | ||||
41 | KIP- | 324420: Add | method to get metrics() in AdminClientSingle Value Fetch in Session Stores | 2. | 12.0 | |
42 | KIP-322: Return new error code for DeleteTopics API when topic deletion disabled.417: Allow JmxTool to connect to a secured RMI port | 2.13.0 | ||||
43 | KIP- | 321: Update TopologyDescription to better represent Source and Sink Nodes415: Incremental Cooperative Rebalancing in Kafka Connect | 2.3 | 2.1.0 | ||
44 | KIP-320: Allow fetchers to detect and handle log truncation414: Expose Embedded ClientIds in Kafka Streams | 2.1.0 (partially implemented) / 2.2.0 | ||||
45 | KIP- | 319: Replace segments with segmentInterval in WindowBytesStoreSupplier412: Extend Admin API to support dynamic application log levels | 2. | 14.0 (WIP) | ||
46 | 2.3.0 | |||||
47 | KIP-402: Improve fairness in SocketServer processors313: Add KStream.flatTransform and KStream.flatTransformValues | 2.2.0 (partially implemented) / 2.3.0 (WIP) | ||||
4748 | KIP-312 Add Overloaded StreamsBuilder Build Method to Accept java.util.Properties401: TransformerSupplier/ProcessorSupplier StateStore connecting | 2.14.0 (WIP) | 48||||
49 | KIP- | 308: Support dynamic update of max.connections.per.ip/max.connections.per.ip.overrides396: Add Commit/List Offsets Operations to AdminClient | 2. | 14.0 (WIP) | 49||
50 | KIP-306: Configuration for Delaying Response to Failed Authentication394: Require member.id for initial join group request | 2.12.0 | 50||||
51 | KIP- | 305: Add Connect primitive number converters393: Time windowed serde to properly deserialize changelog input topic | 2. | 02.0 | ||
5152 | KIP- | 303: Add Dynamic Routing in Streams Sink389: Introduce a configurable consumer group size limit | 2. | 02.0 | ||
5253 | KIP-302 - Enable Kafka clients to use all DNS resolved IP addresses386: Standardize on Min/Avg/Max metrics' default value | 2.12.0 | 53||||
54 | KIP-298: Error Handling in Connect | 2.0.0 | 54 | KIP-297: Externalizing Secrets for Connect Configurations382: MirrorMaker 2.0 | 2.04.0 | |
55 | 295 Add Streams Configuration Allowing for Optional Topology Optimization | 2. | 04.0 (WIP) | |||
56 | KIP-294 - Enable TLS hostname verification by default380: Detect outdated control requests and bounced brokers using broker generation | 2.02.0 | ||||
57 | KIP-292: Add transformValues() method to KTable377: TopicCommand to use AdminClient | 2.02.0 | ||||
58 | KIP-290: Support for Prefixed ACLs | 2.0.0 | 376: Implement AutoClosable on appropriate classes that want to be used in a try-with-resource statement | 2.2.0 | ||
59 | KIP-289: Improve the default group id behavior in KafkaConsumer374: Add '--help' option to all available Kafka CLI commands | 2.2.0 | ||||
60 | KIP- | 285: Connect Rest Extension Plugin372: Naming Repartition Topics for Joins and Grouping | 2. | 01.0 | ||
61 | KIP- | 284: Set default retention ms for Streams repartition topics to Long.MAX_VALUE371: Add a configuration to build custom SSL principal name | 2. | 02.0 | ||
62 | KIP- | 283: Efficient Memory Usage for Down-Conversion369: Alternative Partitioner to Support "Always Round-Robin" Selection | 2. | 04.0 | ||
63 | KIP | -282: Add the listener name to the authentication context368: Allow SASL Connections to Periodically Re-Authenticate | 2. | 02.0 | ||
64 | KIP-281: ConsumerPerformance: Increase Polling Loop Timeout and Make It Reachable by the End User367 Introduce close(Duration) to Producer and AdminClient instead of close(long, TimeUnit) | 2.02.0 | ||||
65 | KIP-279: Fix log divergence between leader and follower after fast leader fail over366: Make FunctionConversions deprecated | 2.01.0 | ||||
66 | KIP-278 - Add version option to Kafka's commands365: Materialized, Serialized, Joined, Consumed and Produced with implicit Serde | 2.01.0 | ||||
67 | KIP-277 - Fine Grained ACL for CreateTopics APIKIP-361: Add Consumer Configuration to Disable Auto Topic Creation | 2.03.0 | ||||
68 | KIP- | 276 - Add StreamsConfig prefix for different consumers360: Improve handling of unknown producer | 2. | 04.0 (WIP) | ||
69 | KIP- | 274: Kafka Streams Skipped Records Metrics359: Verify leader epoch in produce requests | 2. | 04.0 (WIP) | ||
70 | KIP-272358: Add Migrate Streams API version tag to broker's RequestsPerSec metricto Duration instead of long ms times | 2.01.0 | ||||
71 | KIP- | 270 - A Scala Wrapper Library for Kafka Streams357: Add support to list ACLs per principal | 2. | 01.0 | ||
72 | KIP- | 268: Simplify Kafka Streams Rebalance Metadata Upgrade356: Add withCachingDisabled() to StoreBuilder | 2. | 01.0 | ||
73 | KIP- | 267354: Add | Processor Unit Test Support to Kafka Streams Test Utilsa Maximum Log Compaction Lag | 2. | 03.0 | |
74 | KIP-266: Fix consumer indefinite blocking behavior353: Improve Kafka Streams Timestamp Synchronization | 2.01.0 | ||||
75 | KIP- | 265: Make Windowed Serde to public APIs352: Distinguish URPs caused by reassignment | 2. | 04.0 (WIP) | ||
76 | KIP- | 261: Add Single Value Fetch in Window Stores351: Add --under-min-isr option to describe topics command | 2. | 03.0 | ||
77 | KIP-258: Allow to Store Record Timestamps in RocksDB346: Improve LogCleaner behavior on error | 2.3.01 | ||||
78 | KIP- | 257 - Configurable Quota Management345: Introduce static membership protocol to reduce consumer rebalances | 2.4.0 (WIP), partially available in 2.3.0 | |||
79 | KIP-255: OAuth Authentication via SASL/OAUTHBEARER342 Add support for custom SASL extensions in OAuthBearer authentication | 2.01.0 | ||||
80 | KIP- | 251: Allow timestamp manipulation in Processor API341: Update Sticky Assignor's User Data Protocol | 2. | 03.0 | ||
81 | KIP- | 249: Add Delegation Token Operations to KafkaAdminClient340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file | 2. | 01.0 | ||
82 | KIP- | 247: Add public test utils for Kafka Streams339: Create a new IncrementalAlterConfigs API | 2.3.0 | 1.1.0|||
83 | KIP- | 245: Use Properties instead of StreamsConfig in KafkaStreams constructor338 Support to exclude the internal topics in kafka-topics.sh command | 2. | 01.0 | ||
84 | KIP-244: Add Record Header support to Kafka Streams Processor API336: Consolidate ExtendedSerializer/Serializer and ExtendedDeserializer/Deserializer | 2.01.0 | ||||
85 | KIP-243: Make ProducerConfig and ConsumerConfig constructors public332: Update AclCommand to use AdminClient API | 21.1.0 | ||||
86 | KIP- | 239 Add queryableStoreName() to GlobalKTable331 Add default implementation to close() and configure() for Serializer, Deserializer and Serde | 2.3 | 1.1.0 | ||
87 | KIP- | 238: Expose Kafka cluster ID in Connect REST API330: Add retentionPeriod in SessionBytesStoreSupplier | 2 | 1.1.0 | ||
88 | KIP- | 237: More Controller Health Metrics328: Ability to suppress updates for KTables | 2.1.0 (partially implemented) / 2.3.0 (WIP) | |||
89 | KIP-235324: Add DNS alias support for secured connectionmethod to get metrics() in AdminClient | 2.1.0 | ||||
90 | KIP-233: Simplify StreamsBuilder#addGlobalStore322: Return new error code for DeleteTopics API when topic deletion disabled. | 21.1.0 | ||||
91 | KIP-231: Improve the Required ACL of ListGroups API321: Update TopologyDescription to better represent Source and Sink Nodes | 2.1.0 | ||||
92 | KIP- | 229: DeleteGroups API320: Allow fetchers to detect and handle log truncation | 2.1.0 (partially implemented) / 2.2 | 1.1.0 | ||
93 | KIP- | 227 - Introduce Incremental FetchRequests to Increase Partition Scalability319: Replace segments with segmentInterval in WindowBytesStoreSupplier | 2 | 1.1.0 | ||
94 | KIP- | 226 - Dynamic Broker Configuration313: Add KStream.flatTransform and KStream.flatTransformValues | 2.2.0 (partially implemented) / 2.3 | 1.1.0 | ||
95 | KIP- | 225 - Use tags for consumer “records.lag” metrics312 Add Overloaded StreamsBuilder Build Method to Accept java.util.Properties | 2 | 1.1.0 | ||
96 | KIP- | 224309: Add | configuration parameter `retries` to Streams API1.1.0 | toUpperCase support to sasl.kerberos.principal.to.local rule | 2.4.0 | |
97 | KIP-308: Support dynamic update of max.connections.per.ip/max.connections.per.ip.overrides | 2.1 | 97 | KIP-223 - Add per-topic min lead and per-partition lead metrics to KafkaConsumer | 2.0.0 | |
98 | KIP- | 222 - Add Consumer Group operations to Admin API307: Allow to define custom processor names with KStreams DSL | 2. | 03.0 (partial) | ||
99 | KIP- | 220: Add AdminClient into Kafka Streams' ClientSupplier306: Configuration for Delaying Response to Failed Authentication | 2 | 1.1.0 | ||
100 | KIP-219 - Improve quota communication305: Add Connect primitive number converters | 2.0.0 | ||||
101 | KIP- | 218: Make KafkaFuture.Function java 8 lambda compatible303: Add Dynamic Routing in Streams Sink | 2.0 | 1.1.0 | ||
102 | KIP- | 215: Add topic regex support for Connect sinks302 - Enable Kafka clients to use all DNS resolved IP addresses | 2 | 1.1.0 | ||
103 | KIP- | 214300: Add | zookeeper.max.in.flight.requests config to the brokerWindowed KTable API in StreamsBuilder | 2.4 | 1.1.0 | |
104 | KIP- | 212: Enforce set of legal characters for connector names298: Error Handling in Connect | 2.0 | 1.1.0 | ||
105 | KIP- | 211: Revise Expiration Semantics of Consumer Group Offsets297: Externalizing Secrets for Connect Configurations | 2. | 10.0 | ||
106 | KIP- | 210 - Provide for custom error handling when Kafka Streams fails to produce295 Add Streams Configuration Allowing for Optional Topology Optimization | 2.0 | 1.1.0 | ||
107 | KIP- | 208: Add SSL support to Kafka Connect REST interface294 - Enable TLS hostname verification by default | 2.0 | 1.1.0 | ||
108 | 207: Offsets returned by ListOffsetsResponse should be monotonically increasing even during a partition leader change | Accepted2.2.0 | ||||
109 | KIP-206292: Add support for UUID serialization and deserializationtransformValues() method to KTable | 2.10.0 | ||||
110 | KIP- | 205: Add all() and range() API to ReadOnlyWindowStore290: Support for Prefixed ACLs | 2.0 | 1.1.0 | ||
111 | KIP- | 204 : Adding records deletion operation to the new Admin Client API289: Improve the default group id behavior in KafkaConsumer | 2.2 | 1.1.0 | ||
112 | KIP-203: Add toLowerCase support to sasl.kerberos.principal.to.local rule285: Connect Rest Extension Plugin | 2.0 1.1.0 | ||||
113 | KIP- | 202: Move merge() from StreamsBuilder to KStream284: Set default retention ms for Streams repartition topics to Long.MAX_VALUE | 2 | 1.0.0 | ||
114 | KIP- | 198: Remove ZK dependency from Streams Reset Tool283: Efficient Memory Usage for Down-Conversion | 2 | 1.0.0 | ||
115 | KIP- | 197 Connect REST API should include the connector type when describing a connector282: Add the listener name to the authentication context | 2 | 1.0.0 | ||
116 | KIP- | 196: Add metrics to Kafka Connect framework281: ConsumerPerformance: Increase Polling Loop Timeout and Make It Reachable by the End User | 2 | 1.0.0 | ||
117 | KIP-195: AdminClient.createPartitions279: Fix log divergence between leader and follower after fast leader fail over | 21.0.0 | ||||
118 | KIP- | 192 : Provide cleaner semantics when idempotence is enabled278 - Add version option to Kafka's commands | 2 | 1.0.0 | ||
119 | KIP | -191: KafkaConsumer.subscribe() overload that takes just Pattern-277 - Fine Grained ACL for CreateTopics API | 2 | 1.0.0 | ||
120 | KIP-190: Handle client-ids consistently between clients and brokers276 - Add StreamsConfig prefix for different consumers | 21.0.0 | ||||
121 | KIP-189: Improve principal builder interface and add support for SASL274: Kafka Streams Skipped Records Metrics | 21.0.0 | ||||
122 | KIP- | 188 - Add new metrics to support health checks272: Add API version tag to broker's RequestsPerSec metric | 2 | 1.0.0 | ||
123 | KIP- | 187 - Add cumulative count metric for all Kafka rate metrics270 - A Scala Wrapper Library for Kafka Streams | 2 | 1.0.0 | ||
124 | KIP-186: Increase offsets retention default to 7 days268: Simplify Kafka Streams Rebalance Metadata Upgrade | 2.0.0 | ||||
125 | KIP-183 - Change PreferredReplicaLeaderElectionCommand to use AdminClient267: Add Processor Unit Test Support to Kafka Streams Test Utils | 2.20.0 | ||||
126 | KIP- | 182: Reduce Streams DSL overloads and allow easier use of custom storage engines266: Fix consumer indefinite blocking behavior | 2 | 1.0.0 | ||
127 | KIP-180: Add a broker metric specifying the number of consumer group rebalances in progress265: Make Windowed Serde to public APIs | 2.01.1.0 | ||||
128 | KIP-177: Consumer perf tool should count rebalance time261: Add Single Value Fetch in Window Stores | 21.0.0 | ||||
129 | KIP-176: Remove deprecated new-consumer option for tools258: Allow to Store Record Timestamps in RocksDB | 2.03.0 (partially implemented) | ||||
130 | KIP-175: Additional '--describe' views for ConsumerGroupCommand257 - Configurable Quota Management | 2.01.1.0 | ||||
131 | KIP- | 174 - Deprecate and remove internal converter configs in WorkerConfig255: OAuth Authentication via SASL/OAUTHBEARER | 2.0.0 | |||
132 | KIP-173: Add prefix to StreamsConfig to enable setting default internal topic configs251: Allow timestamp manipulation in Processor API | 21.0.0 | ||||
133 | KIP-171 - Extend Consumer Group Reset Offset for Stream Application249: Add Delegation Token Operations to KafkaAdminClient | 2.01.1.0 | ||||
134 | KIP-168247: Add GlobalTopicCount and GlobalPartitionCount metric per clusterpublic test utils for Kafka Streams | 1.01.0 | ||||
135 | KIP-167: Add interface for the state store restoration process245: Use Properties instead of StreamsConfig in KafkaStreams constructor | 21.0.0 | ||||
136 | KIP- | 164 - Add UnderMinIsrPartitionCount and per-partition UnderMinIsr metrics244: Add Record Header support to Kafka Streams Processor API | 2 | 1.0.0 | ||
137 | KIP- | 163: Lower the Minimum Required ACL Permission of OffsetFetch243: Make ProducerConfig and ConsumerConfig constructors public | 1. | 01.0 | ||
138 | KIP-162: Enable topic deletion by default239 Add queryableStoreName() to GlobalKTable | 1.01.0 | ||||
139 | KIP-161: streams deserialization exception handlers238: Expose Kafka cluster ID in Connect REST API | 1.01.0 | ||||
140 | KIP-160: Augment KStream.print(), KStream.writeAsText() to allow users pass in extra parameters in the printed string | 1.0.0 | 237: More Controller Health Metrics | 2.0.0 | ||
141 | KIP-235: Add DNS alias support for secured connection | 2.1 | 141 | KIP-157 - Add consumer config options to streams reset tool | 1.0.0 | |
142 | KIP- | 156 Add option "dry run" to Streams application reset tool233: Simplify StreamsBuilder#addGlobalStore | 1.1 | 0.11.0.0 | ||
143 | KIP-155 - Add range scan for windowed state stores231: Improve the Required ACL of ListGroups API | 2.1.00.11.0.0 | ||||
144 | KIP-154 Add Kafka Connect configuration properties for creating internal topics229: DeleteGroups API | 1.1.00.11.0.0 | ||||
145 | KIP-153: Include only client traffic in BytesOutPerSec metric-227 - Introduce Incremental FetchRequests to Increase Partition Scalability | 1.1.00.11.0.0 | ||||
146 | KIP-152 - Improve diagnostics for SASL authentication failures226 - Dynamic Broker Configuration | 1.01.0 | ||||
147 | KIP-151 Expose Connector type in REST API-225 - Use tags for consumer “records.lag” metrics | 1.10.11.0.0 | ||||
148 | KIP-150 - Kafka-Streams Cogroup224: Add configuration parameter `retries` to Streams API | 1.1.02.3.0 (WIP) | ||||
149 | KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner | 1.1.0 (partially implemented) (WIP for 2.3.0) | -223 - Add per-topic min lead and per-partition lead metrics to KafkaConsumer | 2.0.0 | ||
150 | KIP-222 - Add Consumer Group operations to Admin API | 2 | 150 | KIP-146 - Classloading Isolation in Connect | 0.11.0.0 | |
151 | KIP-145 - Expose Record Headers in Kafka Connect221: Enhance DSL with Connecting Topic Creation and Repartition Hint | 2.4.0 (WIP)1.1.0 | ||||
152 | KIP-144: Exponential backoff for broker reconnect attempts220: Add AdminClient into Kafka Streams' ClientSupplier | 1.1.00.11.0.0 | ||||
153 | KIP-143: Controller Health Metrics219 - Improve quota communication | 2.0.11.0.0 | ||||
154 | KIP-140: Add administrative RPCs for adding, deleting, and listing ACLs218: Make KafkaFuture.Function java 8 lambda compatible | 1.1.00.11.0.0 | ||||
155 | KIP-138: Change punctuate semantics215: Add topic regex support for Connect sinks | 1.01.0 | ||||
156 | KIP-137: Enhance TopicCommand --describe to show topics marked for deletion214: Add zookeeper.max.in.flight.requests config to the broker | 1.1.00.11.0.0 | ||||
157 | KIP- | 136: Add Listener name to SelectorMetrics tags213 Support non-key joining in KTable | 2.4.0 (WIP) | 0.11.0.0|||
158 | KIP-134: Delay initial consumer group rebalance212: Enforce set of legal characters for connector names | 1.10.11.0.0 | ||||
159 | KIP-133: Describe and Alter Configs Admin APIs211: Revise Expiration Semantics of Consumer Group Offsets | 2.10.11.0.0 | ||||
160 | KIP- | 130: Expose states of active tasks to KafkaStreams public API210 - Provide for custom error handling when Kafka Streams fails to produce | 1. | 01.0 | ||
161 | KIP-129: Streams Exactly-Once Semantics208: Add SSL support to Kafka Connect REST interface | 1.10.11.0.0 | ||||
162 | KIP-128: Add ByteArrayConverter for Kafka Connect207: Offsets returned by ListOffsetsResponse should be monotonically increasing even during a partition leader change | 2.2.00.11.0.0 | ||||
163 | KIP- | 126 - Allow KafkaProducer to split and resend oversized batches.206: Add support for UUID serialization and deserialization | 2.1.0 | 0.11.0.0|||
164 | KIP-124 - Request rate quotas | 0.11.0.0 | 205: Add all() and range() API to ReadOnlyWindowStore | 1.1.0 | ||
165 | KIP-123: Allow per stream/table timestamp extractor204 : Adding records deletion operation to the new Admin Client API | 1.1.00.11.0.0 | ||||
166 | KIP-122: Add Reset Consumer Group Offsets tooling203: Add toLowerCase support to sasl.kerberos.principal.to.local rule | 1.10.11.0.0 | ||||
167 | KIP-121: Add KStream peek method202: Move merge() from StreamsBuilder to KStream | 1.0.11.0.0 | ||||
168 | KIP-120: Cleanup Kafka Streams builder API198: Remove ZK dependency from Streams Reset Tool | 1.0.0 | ||||
169 | KIP-197 Connect REST API should include the connector type when describing a connector | 1.119: Drop Support for Scala 2.10 in Kafka 0.110.11.0.0 | ||||
170 | KIP-118: Drop Support for Java 7196: Add metrics to Kafka Connect framework | 12.0.0 | ||||
171 | KIP-117195: Add a public AdminClient API for Kafka admin operationsAdminClient.createPartitions | 10.11.0.0 | ||||
172 | KIP-192 : Provide cleaner semantics when idempotence is enabled | 1.115: Enforce offsets.topic.replication.factor upon __consumer_offsets auto topic creation0.11.0.0 | ||||
173 | KIP-114: KTable state stores and improved semantics191: KafkaConsumer.subscribe() overload that takes just Pattern | 10.11.0.0 | ||||
174 | KIP-113: Support replicas movement between log directories190: Handle client-ids consistently between clients and brokers | 1.10.0 | ||||
175 | KIP-112: Handle disk failure for JBOD189: Improve principal builder interface and add support for SASL | 1.0.0 | ||||
176 | KIP- | 110: Add Codec for ZStandard Compression188 - Add new metrics to support health checks | 1.0 | 2.1.0 | ||
177 | KIP-109: Old Consumer Deprecation-187 - Add cumulative count metric for all Kafka rate metrics | 10.11.0.0 | ||||
178 | KIP-108: Create Topic Policy186: Increase offsets retention default to 7 days | 2.00.10.2.0 | ||||
179 | KIP-107: Add deleteRecordsBefore() API in 183 - Change PreferredReplicaLeaderElectionCommand to use AdminClient | 02.112.0.0 | ||||
180 | KIP-182: Reduce Streams DSL overloads and allow easier use of custom storage engines | 1.106 - Change Default unclean.leader.election.enabled from True to False0.11.0.0 | ||||
181 | KIP-105: Addition of Record Level for Sensors180: Add a broker metric specifying the number of consumer group rebalances in progress | 1.10.10.2.0 | ||||
182 | KIP-104: Granular Sensors for Streams 177: Consumer perf tool should count rebalance time | 1.0.0.10.2.0 | ||||
183 | KIP- | 103: Separation of Internal and External traffic176: Remove deprecated new-consumer option for tools | 2.0. | 0.10.2.0 | ||
184 | KIP-175: Additional '-102 - Add close with timeout for consumers-describe' views for ConsumerGroupCommand | 1.10.10.2.0 | ||||
185 | KIP-101 - Alter Replication Protocol to use Leader Epoch rather than High Watermark for Truncation174 - Deprecate and remove internal converter configs in WorkerConfig | 20.11.0.0 | ||||
186 | KIP-100 - Relax Type constraints in Kafka Streams API173: Add prefix to StreamsConfig to enable setting default internal topic configs | 1.00.10.2.0 | ||||
187 | KIP-99: Add Global Tables to Kafka Streams-171 - Extend Consumer Group Reset Offset for Stream Application | 1.1.0.10.2.0 | ||||
188 | KIP-98 - Exactly Once Delivery and Transactional Messaging168: Add GlobalTopicCount and GlobalPartitionCount metric per cluster | 10.11.0.0 | ||||
189 | KIP-97: Improved Kafka Client RPC Compatibility Policy167: Add interface for the state store restoration process | 1.0.0.10.2.0 | ||||
190 | 96 164 - Add UnderMinIsrPartitionCount and per-partition metrics for in-sync and assigned replica count | 1.0.0.10.2.0 | ||||
191 | 94 Session Windows | 1.0. | 0.10.2.0 | |||
192 | KIP-93: Improve invalid timestamp handling in Kafka Streams162: Enable topic deletion by default | 1.00.10.2.0 | ||||
193 | KIP-92 - Add per partition lag metrics to KafkaConsumer161: streams deserialization exception handlers | 1.00.10.2.0 | ||||
194 | KIP-91 Provide Intuitive User Timeouts in The Producer160: Augment KStream.print(), KStream.writeAsText() to allow users pass in extra parameters in the printed string | 1.02.1.0 | ||||
195 | KIP- | 90 - Remove zkClient dependency from Streams157 - Add consumer config options to streams reset tool | 1.0 | 0.10.2.0 | ||
196 | KIP- | 89: Allow sink connectors to decouple flush and offset commit156 Add option "dry run" to Streams application reset tool | 0. | 1011. | 20.0 | |
197 | KIP | -88: OffsetFetch Protocol Update-155 - Add range scan for windowed state stores | 0. | 1011. | 20.0 | |
198 | KIP-86: Configurable SASL callback handlers154 Add Kafka Connect configuration properties for creating internal topics | 0.112.0.0 | ||||
199 | KIP-85: Dynamic JAAS configuration for Kafka clients153: Include only client traffic in BytesOutPerSec metric | 0.1011.20.0 | ||||
200 | KIP-84: Support SASL SCRAM mechanisms152 - Improve diagnostics for SASL authentication failures | 1.00.10.2.0 | ||||
201 | KIP-82 - Add Record Headers151 Expose Connector type in REST API | 0.11.0.0 | ||||
202 | KIP-81: Bound Fetch memory usage in the consumer-150 - Kafka-Streams Cogroup | 2.24.0 0 (WIP) | ||||
203 | KIP-79 - ListOffsetRequest/ListOffsetResponse v1 and add timestamp search methods to the new consumer149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner | 1.1.0 (partially implemented) (WIP for 2.4.0)0.10.1.0 | ||||
204 | KIP-78: Cluster Id146 - Classloading Isolation in Connect | 0. 1011. 10.0 | ||||
205 | KIP-77: Improve Kafka Streams Join Semantics-145 - Expose Record Headers in Kafka Connect | 1.1.0.10.2.0 | ||||
206 | KIP-75 - Add per-connector Converters144: Exponential backoff for broker reconnect attempts | 0.1011.10.0 | ||||
207 | KIP-74: Add Fetch Response Size Limit in Bytes143: Controller Health Metrics | 0.11.00.10.1.0 | ||||
208 | KIP-73: Replication Quotas140: Add administrative RPCs for adding, deleting, and listing ACLs | 0.1011.10.0 | ||||
209 | KIP-72: Allow putting a bound on memory consumed by Incoming request 138: Change punctuate semantics | 1.0.0 | ||||
210 | KIP-71: Enable log compaction and deletion to co-exist137: Enhance TopicCommand --describe to show topics marked for deletion | 0.1011.10.0 | ||||
211 | KIP-70: Revise Partition Assignment Semantics on New Consumer's Subscription Change136: Add Listener name to SelectorMetrics tags | 0.1011.10.0 | ||||
212 | KIP-67: Queryable state for Kafka Streams134: Delay initial consumer group rebalance | 0.1011.10.0 | ||||
213 | KIP-66: Single Message Transforms for Kafka Connect133: Describe and Alter Configs Admin APIs | 0.0.10.2.0 / 0.11.0.0 | ||||
214 | KIP- | 65130: Expose | timestamps to Connect0.10.1states of active tasks to KafkaStreams public API | 1.0.0 | ||
215 | KIP-63: Unify store and downstream caching in streams129: Streams Exactly-Once Semantics | 0.1011.10.0 | ||||
216 | KIP-62: Allow consumer to send heartbeats from a background thread128: Add ByteArrayConverter for Kafka Connect | 0.1011.10.0 | ||||
217 | KIP-60 - Make Java client classloading more flexible126 - Allow KafkaProducer to split and resend oversized batches. | 0.1011.10.0 | ||||
218 | KIP-58 - Make Log Compaction Point Configurable124 - Request rate quotas | 0.1011.10.0 | ||||
219 | KIP-57 - Interoperable LZ4 Framing123: Allow per stream/table timestamp extractor | 0.1011.0.0 | ||||
220 | KIP-56: Allow cross origin HTTP requests on all HTTP methods122: Add Reset Consumer Group Offsets tooling | 0.1011.0.0 | ||||
221 | KIP-55: Secure Quotas for Authenticated Users121: Add KStream peek method | 0. 1011. 10.0 | ||||
222 | KIP-54: Sticky Partition Assignment Strategy120: Cleanup Kafka Streams builder API | 10.11.0.0 | ||||
223 | KIP-52: Connector Control APIs119: Drop Support for Scala 2.10 in Kafka 0.11 | 0.1011.0.0 | ||||
224 | KIP-51 - List Connectors REST API118: Drop Support for Java 7 | 20.10.0.0 | ||||
225 | KIP-50 - Move Authorizer to o.a.k.common package117: Add a public AdminClient API for Kafka admin operations | 0.11.0.0.10.1.0 | ||||
226 | KIP-48 Delegation token support for Kafka115: Enforce offsets.topic.replication.factor upon __consumer_offsets auto topic creation | 0.11.01.1.0 | ||||
227 | KIP-45 - Standardize all client sequence interaction on j.u.Collection.114: KTable state stores and improved semantics | 0.1011.0.0 | ||||
228 | KIP-43: Kafka SASL enhancements113: Support replicas movement between log directories | 1.1.00.10.0.0 | ||||
229 | KIP-42: Add Producer and Consumer Interceptors112: Handle disk failure for JBOD | 10.10.0.0 | ||||
230 | KIP- | 41: Consumer Max Records110: Add Codec for ZStandard Compression | 2.1.0 | 0.10.0.0|||
231 | KIP-40: ListGroups and DescribeGroup109: Old Consumer Deprecation | 0.911.0.0 | ||||
232 | KIP-38: ZooKeeper Authentication108: Create Topic Policy | 0.910.02.0 | ||||
233 | KIP-36 - Rack aware replica assignment107: Add deleteRecordsBefore() API in AdminClient | 0.1011.0.0 | ||||
234 | 35 - Retrieving protocol version 106 - Change Default unclean.leader.election.enabled from True to False | 0.1011.0.0 | ||||
235 | KIP-33 - Add a time based log index105: Addition of Record Level for Sensors | 0.10.12.0 | ||||
236 | KIP-32 - Add timestamps to Kafka message104: Granular Sensors for Streams | 0.10.02.0 | ||||
237 | KIP-31 - Move to relative offsets in compressed message sets103: Separation of Internal and External traffic | 0.10.02.0 | ||||
238 | KIP-28 102 - Add a processor clientclose with timeout for consumers | 0.10.02.0 | ||||
239 | KIP-26 - Add Kafka Connect framework for data import/export101 - Alter Replication Protocol to use Leader Epoch rather than High Watermark for Truncation | 0.911.0.0 | ||||
240 | KIP-25 - System test improvements100 - Relax Type constraints in Kafka Streams API | 0.910.02.0 | ||||
241 | KIP-22 - Expose a Partitioner interface in the new producer99: Add Global Tables to Kafka Streams | 0.910.02.0 | ||||
242 | KIP-21 - Dynamic Configuration98 - Exactly Once Delivery and Transactional Messaging | 0.911.0.0 (WIP) | ||||
243 | KIP-20 Enable log preallocate to improve consume performance under windows and some old Linux file system97: Improved Kafka Client RPC Compatibility Policy | 0.910.02.0 | ||||
244 | KIP-19 - Add a request timeout to NetworkClient96 - Add per partition metrics for in-sync and assigned replica count | 0.910.02.0 | ||||
245 | KIP-16 - Automated Replica Lag Tuning94 Session Windows | 0.910.02.0 | ||||
246 | KIP-15 - Add a close method with a timeout in the producer93: Improve invalid timestamp handling in Kafka Streams | 0.910.02.0 | ||||
247 | KIP-13 - Quota Design92 - Add per partition lag metrics to KafkaConsumer | 0.910.02.0 | ||||
248 | KIP-12 - Kafka Sasl/Kerberos and SSL implementation91 Provide Intuitive User Timeouts in The Producer | 2.1.00.9.0.0 | ||||
249 | KIP-11 - Kafka Authorizer design90 - Remove zkClient dependency from Streams | 0.910.02.0 | ||||
250 | KIP-8 - Add a flush method to the producer API89: Allow sink connectors to decouple flush and offset commit | 0.910.02.0 | ||||
251 | KIP-4 - Metadata Protocol Changes88: OffsetFetch Protocol Update | 0.10.02.0 | ||||
252 | KIP-4 - Command line and centralized administrative operations86: Configurable SASL callback handlers | 20.9.0.0, 0.10.0.0, 0.10.1.0 | ||||
253 | KIP-3 - Mirror Maker Enhancement85: Dynamic JAAS configuration for Kafka clients | 0.910.02.0 | ||||
254 | KIP-2 - Refactor brokers to allow listening on multiple ports and IPs84: Support SASL SCRAM mechanisms | 0.910.02.0 | ||||
255 | KIP-1 - Remove support of request.required.acks82 - Add Record Headers | 0.911.0.0 |
...
...
KIPs under discussion
Please insert new rows in sorted order (ascending by KIP number). Please move the adopted/accepted KIPs to the "Adopted KIPs" table above.
Dormant/inactive KIPs
Please insert new rows in sorted order (ascending by KIP number).
Discarded KIPs
Please insert new rows in sorted order (ascending by KIP number).
Discarded KIPs
KIP | Comment | ||||||||
---|---|---|---|---|---|---|---|---|---|
KIP-5 - Broker Configuration Management | Superseded by KIP-21 | ||||||||
KIP-7 - Security - IP Filtering | |||||||||
KIP-9 - SSL Support | Adopted via KAFKA-1690 but not via this KIP process | ||||||||
KIP-18 - JBOD Support | Superseded by KIP-112 and KIP-113 | ||||||||
KIP-24 - Remove ISR information from TopicMetadataRequest and add broker level metadata request | |||||||||
KIP-29 - Add an IsrPropagateIntervalMs configuration to KafkaConfig | No longer need after KAFKA-2722 | ||||||||
KIP-34 Add Partitioner Change Listener to Partitioner Interface for Multiple Use Case | Per feedback it is better do this parition() method and avoid Thread Coordination etc. | ||||||||
KIP-80: Kafka Rest Server | |||||||||
KIP-83 - Allow multiple SASL authenticated Java clients in a single JVM process | After KIP-85, no interface changes, reduced to KAFKA-4180 | ||||||||
KIP-116: Add State Store Checkpoint Interval Configuration | Currently not needed as checkpointing can be done on commit interval. | ||||||||
KIP-127: Pluggable JAAS LoginModule configuration for SSL | Similar functionality can be implemented using a custom PrincipalBuilder. | ||||||||
KIP-132: Augment KStream.print to allow extra parameters in the printed string | Duplicated by KIP-160 | ||||||||
KIP-184: Rename LogCleaner and related classes to LogCompactor | A lot of configuration change will cause more trouble and doesn't seems to be worth it | ||||||||
KIP-111: Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker. | Covered by KIP-189 | ||||||||
KIP-147: Add missing type parameters to StateStoreSupplier factories and KGroupedStream/Table methods | Covered by KIP-182 | ||||||||
KIP-165: Extend Interactive Queries for return latest update timestamp per key | Covered by KIP-258 | ||||||||
KIP-170: Enhanced TopicCreatePolicy and introduction of TopicDeletePolicy | Superseded by KIP-201: Rationalising Policy interfaces | ||||||||
KIP 172: Add regular-expression topic support for sink connector | Superseded by KIP-215: Add topic regex support for Connect sinks | KIP-179: Change ReassignPartitionsCommand to use AdminClient | |||||||
KIP 230: Name Windowing Joins | Covered by KIP-372: Naming Repartition Topics for Joins and Grouping | ||||||||
KIP-232: Detect outdated metadata using per-partition leaderEpoch field | Merged into KIP-320: Allow fetchers to detect and handle log truncation | ||||||||
KIP-246: Connect producers and consumers should inherit worker configs | Withdrawn because proposal could not be made backward compatible with existing behavior | ||||||||
KIP-262: Metadata should include number of state stores for task | Rejected since metadata is no longer required. | ||||||||
KIP-263: Allow broker to skip sanity check of inactive segments on broker startup | Withdrawn because the solution we agreed on does not require interface change | ||||||||
KIP-286: producer.send() should not block on metadata update | Withdrawn because the benefit of not having to wait for metadata is probably not worth the complexity added in producer. | ||||||||
KIP-288: [DISCARDED] Consumer.poll() timeout semantic change and new waitForAssignment method | Discarded in deference to KIP-266 | ||||||||
KIP-310: Add a Kafka Source Connector to Kafka Connect | Withdrawn in favor of KIP-382. | ||||||||
KIP-327: Add describe all topics API to AdminClient | Withdrawn in-favor of filter support in Metadata API and KIP-142 | ||||||||
KIP-344: The auto-generated client id should be passed to MetricsReporter | Bug fix so KIP not needed | ||||||||
KIP-364: Remove implicit Materialized, Consumed and Produced | Subsumed by KIP-365 | ||||||||
KIP-404: Add Kafka Connect configuration parameter for disabling WADL output on OPTIONS request | Discarded, reported as a bug by
| ||||||||
KIP-248 - Create New ConfigCommand That Uses The New AdminClient | Discarded by author as it's partly covered in KIP-339 and KIP-422 |
KIP Discussion Recordings
...