This page describes a proposed Kafka Improvement Proposal (KIP) process for proposing a major change to Kafka.
To create your own KIP, click on
Create from template | ||||||||
---|---|---|---|---|---|---|---|---|
|
dev@kafka.apache.org
and request permission (http://kafka.apache.org/contact). Also add an entry to the table KIPs under discussion (for Streams API KIPs, please also add it to Kafka Streams sub page)....
- Click
. Take the next available KIP number and give your proposal a descriptive heading. e.g. "KIP 42: Allow Infinite Retention With Bounded Disk Usage".Create from template templateName 54329345 templateId 54329345 title KIP-NEXT: Insert Title Here buttonLabel Create KIP - Fill in the sections as described above
- Start a [DISCUSS] thread on the Apache mailing list. Please ensure that the subject of the thread is of the format [DISCUSS] KIP-{your KIP number} {your KIP heading} The and the body contains a link to your new KIP. The discussion should happen on the mailing list, not on the wiki, since the wiki comment system doesn't work well for larger discussions. In the process of the discussion you may update the proposal. You should let people know the changes you are making. When you feel you have a finalized proposal
- Once the proposal is finalized call a [VOTE] to have the proposal adopted. These proposals are more serious than code changes and more serious even than release votes. The criteria for acceptance is lazy majority. The vote should remain open for at least 72 hours.
- Please update the KIP wiki page, and the index below, to reflect the current stage of the KIP after a vote. This acts as the permanent record indicating the result of the KIP (e.g., Accepted or Rejected). Also report the result of the KIP vote to the voting thread on the mailing list so the conclusion is clear.
KIP round-up
Next KIP Number: 387529
Use this number as the identifier for your KIP and increment this value.
Adopted KIPs
Please insert new rows in sorted order (descending by KIP number).
KIP (please keep this sorted by KIP number) | Release | ||||||
---|---|---|---|---|---|---|---|
1 | KIP- | 372: Naming Repartition Topics for Joins and Grouping2.1.0500: Replace ZooKeeper with a Self-Managed Metadata Quorum | |||||
2 | KIP-371: Add a configuration to build custom SSL principal name482: The Kafka Protocol should Support Optional Tagged Fields | 2.24.0 (WIP) | |||||
3 | KIP | 368: Allow SASL Connections to Periodically Re-Authenticate-504 - Add new Java Authorizer Interface | 2. | 24.0 (WIP) | |||
4 | KIP-367 Introduce close(Duration) to Producer and AdminClient instead of close(long, TimeUnit)503: Add metric for number of topics marked for deletion | 2.24.0 (WIP) | |||||
5 | KIP- | 366: Make FunctionConversions deprecated497: Add inter-broker API to alter ISR | 2. | 14.0 (WIP) | |||
6 | KIP-365: Materialized, Serialized, Joined, Consumed and Produced with implicit Serde496: Administrative API to delete consumer offsets | 2.14.0 (WIP) | |||||
7 | KIPKIP- | 361: Add Consumer Configuration to Disable Auto Topic Creation495: Dynamically Adjust Log Levels in Connect | 2. | 24.0 (WIP) | |||
8 | KIP-359: Verify leader epoch in produce requests488: Clean up Sum,Count,Total Metrics | 2.24.0 (WIP) | |||||
9 | KIP-358: Migrate Streams API to Duration instead of long ms times484: Expose metrics for group and transaction metadata loading duration | 2.24.0 (WIP) | |||||
10 | KIP- | 357: Add support to list ACLs per principal481: SerDe Improvements for Connect Decimal type in JSON | 2. | 14.0 (WIP) | |||
11 | KIP- | 356: Add withCachingDisabled() to StoreBuilder480: Sticky Partitioner | 2. | 14.0 | |||
12 | KIP- | 353: Improve Kafka Streams Timestamp Synchronization479: Add Materialized to Join | 2. | 14.0(WIP) | |||
13 | KIP-342 Add support for custom SASL extensions in OAuthBearer authentication476: Add Java AdminClient Interface | 2.14.0 | |||||
14 | KIP- | 341: Update Sticky Assignor's User Data Protocol475: New Metrics to Measure Number of Tasks on a Connector | 2. | 24.0 (WIP) | |||
15 | KIP- | 339: Create a new IncrementalAlterConfigs API474: To deprecate WindowStore#put(key, value) | 2. | 24.0 (WIP) | |||
16 | KIP- | 338 Support to exclude the internal topics in kafka-topics.sh command471: Expose RocksDB Metrics in Kafka Streams | 2. | 14.0 (WIP) | |||
17 | KIP-340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file467: Augment ProduceResponse error messaging for specific culprit records | 2.14.0 (WIP) | |||||
18 | KIP-336: Consolidate ExtendedSerializer/Serializer and ExtendedDeserializer/Deserializer465: Add Consolidated Connector Endpoint to Connect REST API | 2.13.0 | |||||
19 | KIP- | 332: Update AclCommand to use AdminClient API464: Defaults for AdminClient#createTopic | 2. | 14.0 | |||
20 | KIP-330: Add retentionPeriod in SessionBytesStoreSupplier462: Use local thread id for KStreams | 2.13.0 | |||||
21 | KIP-328: Ability to suppress updates for KTables461: Improve Replica Fetcher behavior at handling partition failure | 2.23.0 (WIP) | |||||
22 | KIP- | 324: Add method to get metrics() in AdminClient460: Admin Leader Election RPC | 2. | 14.0 | |||
23 | KIP-322: Return new error code for DeleteTopics API when topic deletion disabled.458: Connector Client Config Override Policy | 2.13.0 | |||||
24 | KIP-321: Update TopologyDescription to better represent Source and Sink Nodes455: Create an Administrative API for Replica Reassignment | 2.14.0 (WIP) | |||||
25 | KIP-320: Allow fetchers to detect and handle log truncation454: Expansion of the ConnectClusterState interface | 2.23.0 (WIP) | |||||
26 | KIP-319: Replace segments with segmentInterval in WindowBytesStoreSupplier453: Add close() method to RocksDBConfigSetter | 2.13.0 | |||||
27 | KIP- | 313: Add KStream.flatTransform and KStream.flatTransformValues449: Add connector contexts to log messages in Connect workers | 2. | 23.0 | (WIP)|||
28 | KIP-312 Add Overloaded StreamsBuilder Build Method to Accept java.util.Properties446: Add changelog topic configuration to KTable suppress | 2.14.0 (WIP) | |||||
29 | KIP- | 308: Support dynamic update of max.connections.per.ip/max.connections.per.ip.overrides445: In-memory Session Store | 2. | 13.0 | |||
30 | 306: Configuration for Delaying Response to Failed Authentication | 2.13.0 | |||||
31 | KIP- | 305: Add Connect primitive number converters442: Return to default max poll interval in Streams | 2. | 03.0 | |||
32 | KIP-303: Add Dynamic Routing in Streams Sink440: Extend Connect Converter to support headers | 2.04.0 (WIP) | |||||
33 | KIP-302 - Enable Kafka clients to use all DNS resolved IP addresses436: Add a metric indicating start time | 2.23.0 (WIP) | |||||
34 | KIP-298: Error Handling in Connect434: Add Replica Fetcher and Log Cleaner Count Metrics | 2.04.0 (WIP) | |||||
35 | KIP-297: Externalizing Secrets for Connect Configurations-430 - Return Authorized Operations in Describe Responses | 2.03.0 | |||||
36 | KIP-295 Add Streams Configuration Allowing for Optional Topology Optimization429: Kafka Consumer Incremental Rebalance Protocol | 2.04.0 (WIP) | |||||
37 | KIP- | 294 - Enable TLS hostname verification by default428: Add in-memory window store | 2. | 03.0 | |||
38 | KIP- | 292427: Add | transformValues() method to KTableAtMinIsr topic partition category (new metric & TopicCommand option) | 2. | 03.0 | ||
39 | KIP-290: Support for Prefixed ACLs425: Add some Log4J Kafka Appender Properties for Producing to Secured Brokers | 2.03.0 | |||||
40 | KIP-289: Improve the default group id behavior in KafkaConsumer421: Support resolving externalized secrets in AbstractConfig | 2.23.0 (WIP) | |||||
41 | KIP-285: Connect Rest Extension Plugin420: Add Single Value Fetch in Session Stores | 2.02.0 | |||||
42 | KIP- | 284: Set default retention ms for Streams repartition topics to Long.MAX_VALUE417: Allow JmxTool to connect to a secured RMI port | 2. | 03.0 | |||
43 | KIP- | 283: Efficient Memory Usage for Down-Conversion415: Incremental Cooperative Rebalancing in Kafka Connect | 2. | 03.0 | |||
44 | KIP-282: Add the listener name to the authentication context414: Expose Embedded ClientIds in Kafka Streams | 2.02.0 | |||||
45 | KIP- | 281: ConsumerPerformance: Increase Polling Loop Timeout and Make It Reachable by the End User412: Extend Admin API to support dynamic application log levels | 2. | 04.0 (WIP) | |||
46 | 279: Fix log divergence between leader and follower after fast leader fail over | 2.03.0 | |||||
47 | KIP-278 - Add version option to Kafka's commands402: Improve fairness in SocketServer processors | 2.02.0 (partially implemented) / 2.3.0 | |||||
48 | KIP-401: TransformerSupplier/ProcessorSupplier StateStore connecting | 2.4.0 (WIP) | 48 | KIP-277 - Fine Grained ACL for CreateTopics API | 2.0.0 | ||
49 | KIP- | 276 - Add StreamsConfig prefix for different consumers396: Add Commit/List Offsets Operations to AdminClient | 2. | 04.0 (WIP) | |||
50 | KIP- | 274: Kafka Streams Skipped Records Metrics394: Require member.id for initial join group request | 2. | 02.0 | |||
51 | KIP-272: Add API version tag to broker's RequestsPerSec metric393: Time windowed serde to properly deserialize changelog input topic | 2.02.0 | |||||
52 | KIP- | 270 - A Scala Wrapper Library for Kafka Streams389: Introduce a configurable consumer group size limit | 2. | 02.0 | |||
53 | KIP- | 268: Simplify Kafka Streams Rebalance Metadata Upgrade386: Standardize on Min/Avg/Max metrics' default value | 2. | 02.0 | |||
54 | KIP- | 267: Add Processor Unit Test Support to Kafka Streams Test Utils382: MirrorMaker 2.0 | 2. | 04.0 | |||
55 | 266: Fix consumer indefinite blocking behavior | 2. | 04.0 (WIP) | ||||
56 | KIP- | 265: Make Windowed Serde to public APIs380: Detect outdated control requests and bounced brokers using broker generation | 2. | 02.0 | |||
57 | KIP- | 261: Add Single Value Fetch in Window Stores377: TopicCommand to use AdminClient | 2. | 02.0 | |||
58 | KIP | -257 - Configurable Quota Management-376: Implement AutoClosable on appropriate classes that want to be used in a try-with-resource statement | 2. | 02.0 | |||
59 | KIP-255: OAuth Authentication via SASL/OAUTHBEARER374: Add '--help' option to all available Kafka CLI commands | 2.02.0 | |||||
60 | KIP- | 251: Allow timestamp manipulation in Processor API372: Naming Repartition Topics for Joins and Grouping | 2. | 01.0 | |||
61 | KIP-249371: Add Delegation Token Operations to KafkaAdminClienta configuration to build custom SSL principal name | 2.02.0 | |||||
62 | KIP-247: Add public test utils for Kafka Streams369: Alternative Partitioner to Support "Always Round-Robin" Selection | 2.41.1.0 | |||||
63 | KIP -245: Use Properties instead of StreamsConfig in KafkaStreams constructor368: Allow SASL Connections to Periodically Re-Authenticate | 2.02.0 | |||||
64 | KIP-244: Add Record Header support to Kafka Streams Processor API367 Introduce close(Duration) to Producer and AdminClient instead of close(long, TimeUnit) | 2.02.0 | |||||
65 | KIP-243366: Make ProducerConfig and ConsumerConfig constructors publicFunctionConversions deprecated | 21.1.0 | |||||
66 | KIP- | 239 Add queryableStoreName() to GlobalKTable365: Materialized, Serialized, Joined, Consumed and Produced with implicit Serde | 2 | 1.1.0 | |||
67 | KIPKIP- | 238: Expose Kafka cluster ID in Connect REST API361: Add Consumer Configuration to Disable Auto Topic Creation | 2.3 | 1.1.0 | |||
68 | KIP- | 237: More Controller Health Metrics360: Improve handling of unknown producer | 2. | 04.0 (WIP) | |||
69 | KIP- | 233: Simplify StreamsBuilder#addGlobalStore359: Verify leader epoch in produce requests | 2.4.0 (WIP) | 1.1.0||||
70 | KIP-231: Improve the Required ACL of ListGroups API358: Migrate Streams API to Duration instead of long ms times | 2.1.0 | |||||
71 | KIP-229: DeleteGroups API357: Add support to list ACLs per principal | 21.1.0 | |||||
72 | KIP- | 227 - Introduce Incremental FetchRequests to Increase Partition Scalability356: Add withCachingDisabled() to StoreBuilder | 2 | 1.1.0 | |||
73 | KIP- | 226 - Dynamic Broker Configuration354: Add a Maximum Log Compaction Lag | 2.3 | 1.1.0 | |||
74 | KIP- | 225 - Use tags for consumer “records.lag” metrics353: Improve Kafka Streams Timestamp Synchronization | 2 | 1.1.0 | |||
75 | KIP- | 224: Add configuration parameter `retries` to Streams API352: Distinguish URPs caused by reassignment | 2.4.0 (WIP) | 1.1.0||||
76 | KIP-351: Add - | 223- | Add perunder- | topicmin | lead and per-partition lead metrics to KafkaConsumer-isr option to describe topics command | 2. | 03.0 |
77 | KIP- | 222 - Add Consumer Group operations to Admin API346: Improve LogCleaner behavior on error | 2. | 0.01 | |||
78 | KIP- | 220: Add AdminClient into Kafka Streams' ClientSupplier345: Introduce static membership protocol to reduce consumer rebalances | 2.4.0 (WIP), partially available in 2.3 | 1.1.0 | |||
79 | KIP- | 219 - Improve quota communication342 Add support for custom SASL extensions in OAuthBearer authentication | 2. | 01.0 | |||
80 | KIP- | 218: Make KafkaFuture.Function java 8 lambda compatible341: Update Sticky Assignor's User Data Protocol | 2.3 | 1.1.0 | |||
81 | KIP- | 215: Add topic regex support for Connect sinks340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file | 2 | 1.1.0 | |||
82 | KIP- | 214: Add zookeeper.max.in.flight.requests config to the broker339: Create a new IncrementalAlterConfigs API | 2.3.0 | 1.1.0||||
83 | KIP | -212: Enforce set of legal characters for connector names-338 Support to exclude the internal topics in kafka-topics.sh command | 2 | 1.1.0 | |||
84 | KIP- | 211: Revise Expiration Semantics of Consumer Group Offsets336: Consolidate ExtendedSerializer/Serializer and ExtendedDeserializer/Deserializer | 2.1.0 | ||||
85 | KIP-210 - Provide for custom error handling when Kafka Streams fails to produce332: Update AclCommand to use AdminClient API | 21.1.0 | |||||
86 | KIP- | 208: Add SSL support to Kafka Connect REST interface331 Add default implementation to close() and configure() for Serializer, Deserializer and Serde | 2.3 | 1.1.0 | |||
87 | KIP- | 207: Offsets returned by ListOffsetsResponse should be monotonically increasing even during a partition leader change330: Add retentionPeriod in SessionBytesStoreSupplier | 2. | 21.0 | (WIP)|||
88 | KIP-206: Add support for UUID serialization and deserialization328: Ability to suppress updates for KTables | 2.1.0 (partially implemented) / 2.3.0 (WIP) | |||||
89 | KIP- | 205324: Add | allmethod to get metrics() | and range() API to ReadOnlyWindowStorein AdminClient | 2 | 1.1.0 | |
90 | KIP-204 : Adding records deletion operation to the new Admin Client API322: Return new error code for DeleteTopics API when topic deletion disabled. | 21.1.0 | |||||
91 | KIP- | 203: Add toLowerCase support to sasl.kerberos.principal.to.local rule321: Update TopologyDescription to better represent Source and Sink Nodes | 2 | 1.1.0 | |||
92 | KIP- | 202: Move merge() from StreamsBuilder to KStream320: Allow fetchers to detect and handle log truncation | 2. | 1.0 (partially implemented) / 2.2.0 | |||
93 | KIP- | 198: Remove ZK dependency from Streams Reset Tool319: Replace segments with segmentInterval in WindowBytesStoreSupplier | 2. | 1.0 | .0|||
94 | KIP- | 197 Connect REST API should include the connector type when describing a connector313: Add KStream.flatTransform and KStream.flatTransformValues | 2.2.0 (partially implemented) / 2.3. | 1.0.0 | |||
95 | KIP- | 196: Add metrics to Kafka Connect framework312 Add Overloaded StreamsBuilder Build Method to Accept java.util.Properties | 2. | 1.0 | .0|||
96 | KIP-195: AdminClient.createPartitions308: Support dynamic update of max.connections.per.ip/max.connections.per.ip.overrides | 2.1.0.0 | |||||
97 | KIP- | 192 : Provide cleaner semantics when idempotence is enabled307: Allow to define custom processor names with KStreams DSL | 2.3.0 (partial) | 1.0.0||||
98 | KIP- | 191: KafkaConsumer.subscribe() overload that takes just Pattern306: Configuration for Delaying Response to Failed Authentication | 2. | 1.0 | .0|||
99 | KIP-190: Handle client-ids consistently between clients and brokers305: Add Connect primitive number converters | 21.0.0 | |||||
100 | KIP- | 189: Improve principal builder interface and add support for SASL303: Add Dynamic Routing in Streams Sink | 2 | 1.0.0 | |||
101 | KIP- | 188 - Add new metrics to support health checks302 - Enable Kafka clients to use all DNS resolved IP addresses | 2. | 1.0 | .0|||
102 | KIP- | 187 - Add cumulative count metric for all Kafka rate metrics300: Add Windowed KTable API in StreamsBuilder | 2.4 | 1.0.0 | |||
103 | KIP- | 186: Increase offsets retention default to 7 days298: Error Handling in Connect | 2.0.0 | ||||
104 | KIP- | 183 - Change PreferredReplicaLeaderElectionCommand to use AdminClient297: Externalizing Secrets for Connect Configurations | 2.0.0 | ||||
105 | KIP- | 182: Reduce Streams DSL overloads and allow easier use of custom storage engines295 Add Streams Configuration Allowing for Optional Topology Optimization | 2 | 1.0.0 | |||
106 | KIP- | 180: Add a broker metric specifying the number of consumer group rebalances in progress294 - Enable TLS hostname verification by default | 2.0 | 1.1.0 | |||
107 | 177: Consumer perf tool should count rebalance time | Accepted1.0.0 | |||||
108 | KIP-176: Remove deprecated new-consumer option for tools292: Add transformValues() method to KTable | 2.0.0 | |||||
109 | KIP- | 175: Additional '--describe' views for ConsumerGroupCommand290: Support for Prefixed ACLs | 2.0 | 1.1.0 | |||
110 | KIP- | 174 - Deprecate and remove internal converter configs in WorkerConfig289: Improve the default group id behavior in KafkaConsumer | 2. | 02.0 | |||
111 | KIP-173: Add prefix to StreamsConfig to enable setting default internal topic configs285: Connect Rest Extension Plugin | 21.0.0 | |||||
112 | KIP- | 171 - Extend Consumer Group Reset Offset for Stream Application284: Set default retention ms for Streams repartition topics to Long.MAX_VALUE | 2.0 | 1.1.0 | |||
113 | KIP- | 168: Add GlobalTopicCount and GlobalPartitionCount metric per cluster283: Efficient Memory Usage for Down-Conversion | 2 | 1.0.0 | |||
114 | KIP- | 167282: Add | interface for the state store restoration processthe listener name to the authentication context | 2 | 1.0.0 | ||
115 | KIP- | 164 - Add UnderMinIsrPartitionCount and per-partition UnderMinIsr metrics281: ConsumerPerformance: Increase Polling Loop Timeout and Make It Reachable by the End User | 2 | 1.0.0 | |||
116 | KIP- | 163: Lower the Minimum Required ACL Permission of OffsetFetch279: Fix log divergence between leader and follower after fast leader fail over | 2 | 1.0.0 | |||
117 | KIP | -162: Enable topic deletion by default-278 - Add version option to Kafka's commands | 2 | 1.0.0 | |||
118 | KIP- | 161: streams deserialization exception handlers277 - Fine Grained ACL for CreateTopics API | 2 | 1.0.0 | |||
119 | KIP-276 - Add StreamsConfig prefix for different consumers | 2-160: Augment KStream.print(), KStream.writeAsText() to allow users pass in extra parameters in the printed string1.0.0 | |||||
120 | KIP- | 157 - Add consumer config options to streams reset tool274: Kafka Streams Skipped Records Metrics | 2 | 1.0.0 | |||
121 | KIP-156 Add option "dry run" to Streams application reset tool272: Add API version tag to broker's RequestsPerSec metric | 2.0.11.0.0 | |||||
122 | KIP-155 - Add range scan for windowed state stores270 - A Scala Wrapper Library for Kafka Streams | 2.0.11.0.0 | |||||
123 | KIP-154 Add Kafka Connect configuration properties for creating internal topics268: Simplify Kafka Streams Rebalance Metadata Upgrade | 2.0.11.0.0 | |||||
124 | KIP-153: Include only client traffic in BytesOutPerSec metric267: Add Processor Unit Test Support to Kafka Streams Test Utils | 20.11.0.0 | |||||
125 | KIP- | 152 - Improve diagnostics for SASL authentication failures266: Fix consumer indefinite blocking behavior | 2 | 1.0.0 | |||
126 | KIP-151 Expose Connector type in REST API265: Make Windowed Serde to public APIs | 20.11.0.0 | |||||
127 | KIP-150 - Kafka-Streams Cogroup261: Add Single Value Fetch in Window Stores | 2.2.0 (WIP)0.0 | |||||
128 | KIP- | 149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner258: Allow to Store Record Timestamps in RocksDB | 2.3 | 1.1.0 (partially implemented | )) | ||
129 | KIP-146 - Classloading Isolation in Connect257 - Configurable Quota Management | 20.11.0.0 | |||||
130 | KIP- | 145 - Expose Record Headers in Kafka Connect255: OAuth Authentication via SASL/OAUTHBEARER | 2.0 | 1.1.0 | |||
131 | KIP-144: Exponential backoff for broker reconnect attempts251: Allow timestamp manipulation in Processor API | 2.0.11.0.0 | |||||
132 | KIP-143: Controller Health Metrics249: Add Delegation Token Operations to KafkaAdminClient | 2.0.11.0.0 | |||||
133 | KIP-140247: Add administrative RPCs for adding, deleting, and listing ACLspublic test utils for Kafka Streams | 1.10.11.0.0 | |||||
134 | KIP-138: Change punctuate semantics245: Use Properties instead of StreamsConfig in KafkaStreams constructor | 21.0.0 | |||||
135 | KIP- | 137: Enhance TopicCommand --describe to show topics marked for deletion244: Add Record Header support to Kafka Streams Processor API | 2 | 0.11.0.0 | |||
136 | KIP-136: Add Listener name to SelectorMetrics tags243: Make ProducerConfig and ConsumerConfig constructors public | 1.10.11.0.0 | |||||
137 | KIP-134: Delay initial consumer group rebalance239 Add queryableStoreName() to GlobalKTable | 1.10.11.0.0 | |||||
138 | KIP-133: Describe and Alter Configs Admin APIs238: Expose Kafka cluster ID in Connect REST API | 1.1.00.11.0.0 | |||||
139 | KIP- | 130: Expose states of active tasks to KafkaStreams public API237: More Controller Health Metrics | 2 | 1.0.0 | |||
140 | KIP- | 129: Streams Exactly-Once Semantics235: Add DNS alias support for secured connection | 2.1 | 0.11.0.0 | |||
141 | KIP-128: Add ByteArrayConverter for Kafka Connect233: Simplify StreamsBuilder#addGlobalStore | 1.1.00.11.0.0 | |||||
142 | KIP- | 126 - Allow KafkaProducer to split and resend oversized batches.231: Improve the Required ACL of ListGroups API | 2.1 | 0.11.0.0 | |||
143 | KIP-124 - Request rate quotas229: DeleteGroups API | 1.10.11.0.0 | |||||
144 | KIP-123: Allow per stream/table timestamp extractor227 - Introduce Incremental FetchRequests to Increase Partition Scalability | 1.1.00.11.0.0 | |||||
145 | KIP-122: Add Reset Consumer Group Offsets tooling226 - Dynamic Broker Configuration | 1.10.11.0.0 | |||||
146 | KIP-121: Add KStream peek method225 - Use tags for consumer “records.lag” metrics | 1.1.00.11.0.0 | |||||
147 | KIP-120: Cleanup Kafka Streams builder 224: Add configuration parameter `retries` to Streams API | 1.01.0 | |||||
148 | KIP-223 - Add per-topic min lead and per-partition lead metrics to KafkaConsumer | 2-119: Drop Support for Scala 2.10 in Kafka 0.110.11.0.0 | |||||
149 | KIP-118: Drop Support for Java 7-222 - Add Consumer Group operations to Admin API | 2.0.0 | |||||
150 | KIP-117: Add a public AdminClient API for Kafka admin operations221: Enhance DSL with Connecting Topic Creation and Repartition Hint | 2.4.0 (WIP)0.11.0.0 | |||||
151 | KIP-115: Enforce offsets.topic.replication.factor upon __consumer_offsets auto topic 220: Add AdminClient into Kafka Streams' ClientSupplier | 1.1 creation0.11.0.0 | |||||
152 | KIP-114: KTable state stores and improved semantics-219 - Improve quota communication | 20.11.0.0 | |||||
153 | KIP-113: Support replicas movement between log directories218: Make KafkaFuture.Function java 8 lambda compatible | 1.1.0 | |||||
154 | KIP-112: Handle disk failure for JBOD215: Add topic regex support for Connect sinks | 1.01.0 | |||||
155 | KIP- | 110: Add Codec for ZStandard Compression214: Add zookeeper.max.in.flight.requests config to the broker | 1 | 2.1.0 | |||
156 | KIP | -109: Old Consumer Deprecation-213 Support non-key joining in KTable | 2.4.0 (WIP) | 0.11.0.0||||
157 | KIP-108: Create Topic Policy212: Enforce set of legal characters for connector names | 1.10.10.2.0 | |||||
158 | KIP-107: Add deleteRecordsBefore() API in AdminClient211: Revise Expiration Semantics of Consumer Group Offsets | 2.1.00.11.0.0 | |||||
159 | KIP-106 - Change Default unclean.leader.election.enabled from True to False210 - Provide for custom error handling when Kafka Streams fails to produce | 1.1.00.11.0.0 | |||||
160 | KIP-105: Addition of Record Level for Sensors208: Add SSL support to Kafka Connect REST interface | 1.10.10.2.0 | |||||
161 | KIP-104: Granular Sensors for Streams 207: Offsets returned by ListOffsetsResponse should be monotonically increasing even during a partition leader change | 20.10.2.0 | |||||
162 | KIP- | 103: Separation of Internal and External traffic206: Add support for UUID serialization and deserialization | 2.1 | 0.10.2.0 | |||
163 | KIP-102 - Add close with timeout for consumers205: Add all() and range() API to ReadOnlyWindowStore | 1.1.0.10.2.0 | |||||
164 | KIP-101 - Alter Replication Protocol to use Leader Epoch rather than High Watermark for Truncation204 : Adding records deletion operation to the new Admin Client API | 1.10.11.0.0 | |||||
165 | KIP-100 - Relax Type constraints in Kafka Streams API203: Add toLowerCase support to sasl.kerberos.principal.to.local rule | 1.10.10.2.0 | |||||
166 | KIP-99: Add Global Tables to Kafka Streams202: Move merge() from StreamsBuilder to KStream | 1.0.0.10.2.0 | |||||
167 | KIP-98 - Exactly Once Delivery and Transactional Messaging198: Remove ZK dependency from Streams Reset Tool | 1.0.11.0.0 | |||||
168 | KIP-97: Improved Kafka Client RPC Compatibility Policy197 Connect REST API should include the connector type when describing a connector | 1.0.0.10.2.0 | |||||
169 | KIP-96 - 196: Add per partition metrics for in-sync and assigned replica countmetrics to Kafka Connect framework | 1.00.10.2.0 | |||||
170 | KIP-94 Session Windows195: AdminClient.createPartitions | 1.00.10.2.0 | |||||
171 | KIP-93: Improve invalid timestamp handling in Kafka Streams192 : Provide cleaner semantics when idempotence is enabled | 1.0.0.10.2.0 | |||||
172 | KIP-92 - Add per partition lag metrics to KafkaConsumer191: KafkaConsumer.subscribe() overload that takes just Pattern | 1.0.0.10.2.0 | |||||
173 | KIP-91 Provide Intuitive User Timeouts in The Producer-190: Handle client-ids consistently between clients and brokers | 1.02.1.0 | |||||
174 | KIP-90 - Remove zkClient dependency from Streams189: Improve principal builder interface and add support for SASL | 1.0.0.10.2.0 | |||||
175 | KIP-89: Allow sink connectors to decouple flush and offset commit-188 - Add new metrics to support health checks | 1.0.0.10.2.0 | |||||
176 | KIP-88: OffsetFetch Protocol Update-187 - Add cumulative count metric for all Kafka rate metrics | 1.0.0.10.2.0 | |||||
177 | KIP-86: Configurable SASL callback handlers186: Increase offsets retention default to 7 days | 2.0.0 | |||||
178 | KIP-85: Dynamic JAAS configuration for Kafka clients-183 - Change PreferredReplicaLeaderElectionCommand to use AdminClient | 20.10.2.0 | |||||
179 | KIP-84: Support SASL SCRAM mechanisms182: Reduce Streams DSL overloads and allow easier use of custom storage engines | 1.00.10.2.0 | |||||
180 | KIP-82 - Add Record Headers180: Add a broker metric specifying the number of consumer group rebalances in progress | 1.1.00.11.0.0 | |||||
181 | KIP-81: Bound Fetch memory usage in the consumer177: Consumer perf tool should count rebalance time | 1.0.02.2.0 (WIP) | |||||
182 | KIP- | 79 - ListOffsetRequest/ListOffsetResponse v1 and add timestamp search methods to the new consumer176: Remove deprecated new-consumer option for tools | 2.0 | 0.10.1.0 | |||
183 | KIP-78: Cluster Id175: Additional '--describe' views for ConsumerGroupCommand | 1.0.10.1.0 | |||||
184 | KIP-77: Improve Kafka Streams Join Semantics-174 - Deprecate and remove internal converter configs in WorkerConfig | 2.0.0.10.2.0 | |||||
185 | KIP-75 - Add per-connector Converters173: Add prefix to StreamsConfig to enable setting default internal topic configs | 1.0.0.10.1.0 | |||||
186 | KIP-74: Add Fetch Response Size Limit in Bytes171 - Extend Consumer Group Reset Offset for Stream Application | 1.0.10.1.0 | |||||
187 | KIP-73: Replication Quotas168: Add GlobalTopicCount and GlobalPartitionCount metric per cluster | 1.00.10.1.0 | |||||
188 | KIP-72: Allow putting a bound on memory consumed by Incoming request 167: Add interface for the state store restoration process | 1.0.0 | |||||
189 | -71: Enable log compaction and deletion to co-exist -164 - Add UnderMinIsrPartitionCount and per-partition UnderMinIsr metrics | 1.0.0.10.1.0 | |||||
190 | 70: Revise Partition Assignment Semantics on New Consumer's Subscription Change | 1.0. | 0.10.1.0 | ||||
191 | KIP-67: Queryable state for Kafka Streams162: Enable topic deletion by default | 1.0.0.10.1.0 | |||||
192 | KIP-66: Single Message Transforms for Kafka Connect161: streams deserialization exception handlers | 10.10.2.0 / 0.11.0.0 | |||||
193 | KIP-65: Expose timestamps to Connect160: Augment KStream.print(), KStream.writeAsText() to allow users pass in extra parameters in the printed string | 1.00.10.1.0 | |||||
194 | KIP | -63: Unify store and downstream caching in streams-157 - Add consumer config options to streams reset tool | 1.0. | 0.10.1.0 | |||
195 | KIP- | 62: Allow consumer to send heartbeats from a background thread156 Add option "dry run" to Streams application reset tool | 0. | 1011. | 10.0 | ||
196 | KIP- | 60 - Make Java client classloading more flexible155 - Add range scan for windowed state stores | 0. | 1011. | 10.0 | ||
197 | KIP-58 - Make Log Compaction Point Configurable154 Add Kafka Connect configuration properties for creating internal topics | 0.1011.10.0 | |||||
198 | KIP-57 - Interoperable LZ4 Framing153: Include only client traffic in BytesOutPerSec metric | 0.1011.0.0 | |||||
199 | KIP-56: Allow cross origin HTTP requests on all HTTP methods-152 - Improve diagnostics for SASL authentication failures | 10.10.0.0 | |||||
200 | KIP-55: Secure Quotas for Authenticated Users151 Expose Connector type in REST API | 0.1011.10.0 | |||||
201 | KIP-54: Sticky Partition Assignment Strategy150 - Kafka-Streams Cogroup | 2.4.0 (WIP)0.11.0.0 | |||||
202 | KIP-52: Connector Control APIs149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner | 1.1.0 (partially implemented) (WIP for 2.4.0)0.10.0.0 | |||||
203 | KIP-51 - List Connectors REST API146 - Classloading Isolation in Connect | 0. 1011.0.0 | |||||
204 | KIP-50 - Move Authorizer to o.a.k.common package | 0.10.1.0 | 145 - Expose Record Headers in Kafka Connect | 1.1.0 | |||
205 | KIP-48 Delegation token support for Kafka144: Exponential backoff for broker reconnect attempts | 0.11.01.1.0 | |||||
206 | KIP-45 - Standardize all client sequence interaction on j.u.Collection.143: Controller Health Metrics | 0.1011.0.0 | |||||
207 | KIP-43: Kafka SASL enhancements140: Add administrative RPCs for adding, deleting, and listing ACLs | 0.1011.0.0 | |||||
208 | KIP-42: Add Producer and Consumer Interceptors138: Change punctuate semantics | 10.10.0.0 | |||||
209 | KIP-41: Consumer Max Records137: Enhance TopicCommand --describe to show topics marked for deletion | 0.1011.0.0 | |||||
210 | KIP-40: ListGroups and DescribeGroup136: Add Listener name to SelectorMetrics tags | 0.911.0.0 | |||||
211 | KIP-38: ZooKeeper Authentication134: Delay initial consumer group rebalance | 0.911.0.0 | |||||
212 | KIP-36 - Rack aware replica assignment133: Describe and Alter Configs Admin APIs | 0.1011.0.0 | |||||
213 | KIP- | 35 - Retrieving protocol version130: Expose states of active tasks to KafkaStreams public API | 1 | 0.10.0.0 | |||
214 | KIP-33 - Add a time based log index129: Streams Exactly-Once Semantics | 0.1011.10.0 | |||||
215 | KIP-32 - Add timestamps to Kafka message128: Add ByteArrayConverter for Kafka Connect | 0.1011.0.0 | |||||
216 | KIP-31 126 - Move to relative offsets in compressed message setsAllow KafkaProducer to split and resend oversized batches. | 0.1011.0.0 | |||||
217 | KIP-28 - Add a processor client124 - Request rate quotas | 0.1011.0.0 | |||||
218 | KIP-26 - Add Kafka Connect framework for data import/export123: Allow per stream/table timestamp extractor | 0.911.0.0 | |||||
219 | KIP-25 - System test improvements122: Add Reset Consumer Group Offsets tooling | 0.911.0.0 | |||||
220 | KIP-22 - Expose a Partitioner interface in the new producer121: Add KStream peek method | 0. 911.0.0 | |||||
221 | KIP-21 - Dynamic Configuration120: Cleanup Kafka Streams builder API | 1.0.9.0.0 (WIP) | |||||
222 | KIP-20 Enable log preallocate to improve consume performance under windows and some old Linux file system119: Drop Support for Scala 2.10 in Kafka 0.11 | 0.911.0.0 | |||||
223 | KIP-19 - Add a request timeout to NetworkClient118: Drop Support for Java 7 | 2.0.9.0.0 | |||||
224 | KIP-16 - Automated Replica Lag Tuning117: Add a public AdminClient API for Kafka admin operations | 0.911.0.0 | |||||
225 | KIP-15 - Add a close method with a timeout in the producer115: Enforce offsets.topic.replication.factor upon __consumer_offsets auto topic creation | 0.911.0.0 | |||||
226 | KIP-13 - Quota Design114: KTable state stores and improved semantics | 0.911.0.0 | |||||
227 | KIP-12 - Kafka Sasl/Kerberos and SSL implementation113: Support replicas movement between log directories | 1.10.9.0.0 | |||||
228 | KIP-11 - Kafka Authorizer design112: Handle disk failure for JBOD | 1.0.9.0.0 | |||||
229 | KIP- | 8 - Add a flush method to the producer API110: Add Codec for ZStandard Compression | 2.1.0 | 0.9.0.0||||
230 | KIP-4 - Metadata Protocol Changes109: Old Consumer Deprecation | 0.1011.0.0 | |||||
231 | KIP-4 - Command line and centralized administrative operations108: Create Topic Policy | 0.9.0.0, 0.10.0.0, 0.10.1.2.0 | |||||
232 | KIP-3 - Mirror Maker Enhancement107: Add deleteRecordsBefore() API in AdminClient | 0.911.0.0 | |||||
233 | 2 - Refactor brokers to allow listening on multiple ports and IPs 106 - Change Default unclean.leader.election.enabled from True to False | 0.911.0.0 | |||||
234 | KIP-1 - Remove support of request.required.acks105: Addition of Record Level for Sensors | 0.910.02.0 |
...
...
KIPs under discussion
Please insert new rows in sorted order (ascending by KIP number). Please move the adopted/accepted KIPs to the "Adopted KIPs" table above.
Dormant/inactive KIPs
Please insert new rows in sorted order (ascending by KIP number).
Discarded KIPs
Please insert new rows in sorted order (ascending by KIP number).
...
Discarded KIPs
KIP | Comment |
---|---|
KIP-5 - Broker Configuration Management | Superseded by KIP-21 |
KIP-7 - Security - IP Filtering | |
KIP-9 - SSL Support | Adopted via KAFKA-1690 but not via this KIP process |
KIP-18 - JBOD Support | Superseded by KIP-112 and KIP-113 |
KIP-24 - Remove ISR information from TopicMetadataRequest and add broker level metadata request | |
KIP-29 - Add an IsrPropagateIntervalMs configuration to KafkaConfig | No longer need after KAFKA-2722 |
KIP-34 Add Partitioner Change Listener to Partitioner Interface for Multiple Use Case | Per feedback it is better do this parition() method and avoid Thread Coordination etc. |
KIP-80: Kafka Rest Server | |
KIP-83 - Allow multiple SASL authenticated Java clients in a single JVM process | After KIP-85, no interface changes, reduced to KAFKA-4180 |
KIP-116: Add State Store Checkpoint Interval Configuration | Currently not needed as checkpointing can be done on commit interval. |
KIP-127: Pluggable JAAS LoginModule configuration for SSL | Similar functionality can be implemented using a custom PrincipalBuilder. |
KIP-132: Augment KStream.print to allow extra parameters in the printed string | Duplicated by KIP-160 |
KIP-184: Rename LogCleaner and related classes to LogCompactor | A lot of configuration change will cause more trouble and doesn't seems to be worth it |
KIP-111: Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker. | Covered by KIP-189 |
KIP-147: Add missing type parameters to StateStoreSupplier factories and KGroupedStream/Table methods | Covered by KIP-182 |
KIP-170: Enhanced TopicCreatePolicy and introduction of TopicDeletePolicy | Superseded by KIP-201: Rationalising Policy interfaces |
KIP 172: Add regular-expression topic support for sink connector | Superseded by KIP-215: Add topic regex support for Connect sinks | KIP-179: Change ReassignPartitionsCommand to use AdminClient |
KIP 230: Name Windowing Joins | Covered by KIP-372: Naming Repartition Topics for Joins and Grouping |
KIP-232: Detect outdated metadata using per-partition leaderEpoch field | Merged into KIP-320: Allow fetchers to detect and handle log truncation |
KIP-246: Connect producers and consumers should inherit worker configs | Withdrawn because proposal could not be made backward compatible with existing behavior |
KIP-263: Allow broker to skip sanity check of inactive segments on broker startup | Withdrawn because the solution we agreed on does not require interface change |
KIP-286: producer.send() should not block on metadata update | Withdrawn because the benefit of not having to wait for metadata is probably not worth the complexity added in producer. |
KIP-288: [DISCARDED] Consumer.poll() timeout semantic change and new waitForAssignment method | Discarded in deference to KIP-266 |
KIP-327: Add describe all topics API to AdminClient | Withdrawn in-favor of filter support in Metadata API and KIP-142 |
KIP-344: The auto-generated client id should be passed to MetricsReporter | Bug fix so KIP not needed |
KIP-364: Remove implicit Materialized, Consumed and Produced | Subsumed by KIP-365 |
KIP Discussion Recordings
...