This page describes a proposed Kafka Improvement Proposal (KIP) process for proposing a major change to Kafka.
To create your own KIP, click on
| Create from template | ||||||||
|---|---|---|---|---|---|---|---|---|
|
dev@kafka.apache.org and request permission (http://kafka.apache.org/contact). Also add an entry to the table KIPs under discussion (for Streams API KIPs, please also add it to Kafka Streams sub page)....
- Click
. Take the next available KIP number and give your proposal a descriptive heading. e.g. "KIP 42: Allow Infinite Retention With Bounded Disk Usage".Create from template templateName 54329345 templateId 54329345 title KIP-NEXT: Insert Title Here buttonLabel Create KIP - Fill in the sections as described above
- Start a [DISCUSS] thread on the Apache mailing list. Please ensure that the subject of the thread is of the format [DISCUSS] KIP-{your KIP number} {your KIP heading} The and the body contains a link to your new KIP. The discussion should happen on the mailing list, not on the wiki, since the wiki comment system doesn't work well for larger discussions. In the process of the discussion you may update the proposal. You should let people know the changes you are making. When you feel you have a finalized proposal
- Once the proposal is finalized call a [VOTE] to have the proposal adopted. These proposals are more serious than code changes and more serious even than release votes. The criteria for acceptance is lazy majority. The vote should remain open for at least 72 hours.
- Please update the KIP wiki page, and the index below, to reflect the current stage of the KIP after a vote. This acts as the permanent record indicating the result of the KIP (e.g., Accepted or Rejected). Also report the result of the KIP vote to the voting thread on the mailing list so the conclusion is clear.
KIP round-up
Next KIP Number: 411527
Use this number as the identifier for your KIP and increment this value.
Adopted KIPs
Please insert new rows in sorted order (descending by KIP number).
| KIP (please keep this sorted by KIP number) | Release | |||||
|---|---|---|---|---|---|---|
| 1 | KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum | |||||
| 2 | KIP- | 394: Require member.id for initial join group request482: The Kafka Protocol should Support Optional Tagged Fields | 2. | 24.0 (WIP) | ||
| 23 | KIP-386: Standardize on Min/Avg/Max metrics' default value-504 - Add new Java Authorizer Interface | 2.24.0 (WIP) | ||||
| 34 | KIP-380: Detect outdated control requests and bounced brokers using broker generation503: Add metric for number of topics marked for deletion | 2.2.04 | KIP-377: TopicCommand to use AdminClient | 2.2.0 (WIP) | ||
| 5 | KIP-376: Implement AutoClosable on appropriate classes that want to be used in a try-with-resource statement497: Add inter-broker API to alter ISR | 2.24.0 (WIP) | ||||
| 6 | KIP-372: Naming Repartition Topics for Joins and Grouping496: Administrative API to delete consumer offsets | 2.14.0 (WIP) | ||||
| 7 | KIP-371: Add a configuration to build custom SSL principal name495: Dynamically Adjust Log Levels in Connect | 2.24.0 (WIP) | ||||
| 8 | KIP | 368: Allow SASL Connections to Periodically Re-Authenticate-488: Clean up Sum,Count,Total Metrics | 2. | 24.0 | (WIP)||
| 9 | KIP-367 Introduce close(Duration) to Producer and AdminClient instead of close(long, TimeUnit)484: Expose metrics for group and transaction metadata loading duration | 2.24.0 (WIP) | ||||
| 10 | KIP- | 366: Make FunctionConversions deprecated481: SerDe Improvements for Connect Decimal type in JSON | 2. | 14.0 (WIP) | ||
| 11 | KIP- | 365: Materialized, Serialized, Joined, Consumed and Produced with implicit Serde480: Sticky Partitioner | 2. | 14.0 | ||
| 12 | KIPKIP- | 361479: Add | Consumer Configuration to Disable Auto Topic CreationMaterialized to Join | 2. | 24.0(WIP) | |
| 13 | KIP-359: Verify leader epoch in produce requests476: Add Java AdminClient Interface | 2.24.0 (WIP) | ||||
| 14 | KIP- | 358: Migrate Streams API to Duration instead of long ms times475: New Metrics to Measure Number of Tasks on a Connector | 2. | 14.0 (WIP) | ||
| 15 | KIP-357: Add support to list ACLs per principal474: To deprecate WindowStore#put(key, value) | 2.14.0 (WIP) | ||||
| 16 | KIP- | 356: Add withCachingDisabled() to StoreBuilder471: Expose RocksDB Metrics in Kafka Streams | 2. | 14.0 (WIP) | ||
| 17 | KIP-353: Improve Kafka Streams Timestamp Synchronization467: Augment ProduceResponse error messaging for specific culprit records | 2.14.0 (WIP) | ||||
| 18 | KIP-351465: Add --under-min-isr option to describe topics commandConsolidated Connector Endpoint to Connect REST API | 2.23.0 (WIP) | ||||
| 19 | KIP-346: Improve LogCleaner behavior on error | 2 | 464: Defaults for AdminClient#createTopic | 2.4.0 | ||
| 20 | KIP-342 Add support for custom SASL extensions in OAuthBearer authentication462: Use local thread id for KStreams | 2.13.0 | ||||
| 21 | KIP-341: Update Sticky Assignor's User Data Protocol461: Improve Replica Fetcher behavior at handling partition failure | 2.23.0 (WIP) | ||||
| 22 | KIP- | 340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file460: Admin Leader Election RPC | 2.4.0 | |||
| 23 | KIP-458: Connector Client Config Override Policy | 2.3.0 | ||||
| 24 | KIP-455: Create an Administrative API for Replica Reassignment | 2.4 | 2.1.0 | 23 | KIP-339: Create a new IncrementalAlterConfigs API | 2.2.0 (WIP) | 24
| 25 | KIP-338 Support to exclude the internal topics in kafka-topics.sh command454: Expansion of the ConnectClusterState interface | 2.13.0 | ||||
| 2526 | KIP-336: Consolidate ExtendedSerializer/Serializer and ExtendedDeserializer/Deserializer453: Add close() method to RocksDBConfigSetter | 2.13.0 | ||||
| 2627 | KIP-332: Update AclCommand to use AdminClient API449: Add connector contexts to log messages in Connect workers | 2.13.0 | ||||
| 2728 | KIP-331 Add default implementation to close() and configure() for Serializer, Deserializer and Serde446: Add changelog topic configuration to KTable suppress | 2.24.0 (WIP) | 28||||
| 29 | KIP-330: Add retentionPeriod in SessionBytesStoreSupplier | 7 | 29 | KIP-328: Ability to suppress updates for KTables445: In-memory Session Store | 2.23.0 (WIP) | |
| 30 | 324 Add method to get metrics() in AdminClient | 2.13.0 | ||||
| 31 | KIP- | 322: Return new error code for DeleteTopics API when topic deletion disabled.442: Return to default max poll interval in Streams | 2. | 13.0 | ||
| 32 | KIP-321: Update TopologyDescription to better represent Source and Sink Nodes440: Extend Connect Converter to support headers | 2.14.0 (WIP) | ||||
| 33 | KIP-320: Allow fetchers to detect and handle log truncation436: Add a metric indicating start time | 2.13.0 | ||||
| 34 | KIP-319: Replace segments with segmentInterval in WindowBytesStoreSupplier434: Add Replica Fetcher and Log Cleaner Count Metrics | 2.14.0 (WIP) | ||||
| 35 | KIP-313: Add KStream.flatTransform and KStream.flatTransformValues -430 - Return Authorized Operations in Describe Responses | 2.23.0 (WIP) | ||||
| 36 | KIP-312 Add Overloaded StreamsBuilder Build Method to Accept java.util.Properties429: Kafka Consumer Incremental Rebalance Protocol | 2.14.0 (WIP) | ||||
| 37 | KIP-308: Support dynamic update of max.connections.per.ip/max.connections.per.ip.overrides428: Add in-memory window store | 2.13.0 | ||||
| 38 | KIP- | 306: Configuration for Delaying Response to Failed Authentication427: Add AtMinIsr topic partition category (new metric & TopicCommand option) | 2. | 13.0 | ||
| 39 | KIP- | 305425: Add | Connect primitive number converterssome Log4J Kafka Appender Properties for Producing to Secured Brokers | 2. | 03.0 | |
| 40 | KIP-303: Add Dynamic Routing in Streams Sink421: Support resolving externalized secrets in AbstractConfig | 2.03.0 | ||||
| 41 | KIP- | 302 - Enable Kafka clients to use all DNS resolved IP addresses420: Add Single Value Fetch in Session Stores | 2. | 12.0 | ||
| 42 | KIP- | 298: Error Handling in Connect417: Allow JmxTool to connect to a secured RMI port | 2. | 03.0 | ||
| 43 | KIP- | 297: Externalizing Secrets for Connect Configurations415: Incremental Cooperative Rebalancing in Kafka Connect | 2. | 03.0 | ||
| 44 | KIP-295 Add Streams Configuration Allowing for Optional Topology Optimization414: Expose Embedded ClientIds in Kafka Streams | 2.02.0 | ||||
| 45 | KIP- | 294 - Enable TLS hostname verification by default412: Extend Admin API to support dynamic application log levels | 2. | 04.0 (WIP) | ||
| 46 | 292: Add transformValues() method to KTable | 2. | 03.0 | |||
| 47 | KIP-290: Support for Prefixed ACLs402: Improve fairness in SocketServer processors | 2.2.0 (partially implemented) / 2.3.0 | ||||
| 48 | KIP-289: Improve the default group id behavior in KafkaConsumer401: TransformerSupplier/ProcessorSupplier StateStore connecting | 2.24.0 (WIP) | ||||
| 49 | KIP- | 285: Connect Rest Extension Plugin396: Add Commit/List Offsets Operations to AdminClient | 2. | 04.0 (WIP) | ||
| 50 | KIP-284: Set default retention ms for Streams repartition topics to Long.MAX_VALUE394: Require member.id for initial join group request | 2.02.0 | ||||
| 51 | KIP-283: Efficient Memory Usage for Down-Conversion393: Time windowed serde to properly deserialize changelog input topic | 2.02.0 | ||||
| 52 | KIP- | 282: Add the listener name to the authentication context389: Introduce a configurable consumer group size limit | 2. | 02.0 | ||
| 53 | KIP-281: ConsumerPerformance: Increase Polling Loop Timeout and Make It Reachable by the End User386: Standardize on Min/Avg/Max metrics' default value | 2.02.0 | ||||
| 54 | KIP- | 279: Fix log divergence between leader and follower after fast leader fail over382: MirrorMaker 2.0 | 2. | 04.0 | ||
| 55 | 278 - Add version option to Kafka's commands | 2. | 04.0 (WIP) | |||
| 56 | KIP-277 - Fine Grained ACL for CreateTopics API380: Detect outdated control requests and bounced brokers using broker generation | 2.02.0 | ||||
| 57 | KIP- | 276 - Add StreamsConfig prefix for different consumers377: TopicCommand to use AdminClient | 2. | 02.0 | ||
| 58 | KIP- | 274: Kafka Streams Skipped Records Metrics376: Implement AutoClosable on appropriate classes that want to be used in a try-with-resource statement | 2. | 02.0 | ||
| 59 | KIP-272374: Add API version tag to broker's RequestsPerSec metric'--help' option to all available Kafka CLI commands | 2.02.0 | ||||
| 60 | KIP-270 - A Scala Wrapper Library for Kafka Streams372: Naming Repartition Topics for Joins and Grouping | 2.01.0 | ||||
| 61 | KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade371: Add a configuration to build custom SSL principal name | 2.02.0 | ||||
| 62 | KIP-267: Add Processor Unit Test Support to Kafka Streams Test Utils369: Alternative Partitioner to Support "Always Round-Robin" Selection | 2.04.0 | ||||
| 63 | KIP | -266: Fix consumer indefinite blocking behavior368: Allow SASL Connections to Periodically Re-Authenticate | 2. | 02.0 | ||
| 64 | KIP- | 265: Make Windowed Serde to public APIs367 Introduce close(Duration) to Producer and AdminClient instead of close(long, TimeUnit) | 2. | 02.0 | ||
| 65 | KIP-261: Add Single Value Fetch in Window Stores366: Make FunctionConversions deprecated | 2.01.0 | ||||
| 66 | KIP- | 257 - Configurable Quota Management365: Materialized, Serialized, Joined, Consumed and Produced with implicit Serde | 2. | 01.0 | ||
| 67 | KIPKIP-255: OAuth Authentication via SASL/OAUTHBEARER361: Add Consumer Configuration to Disable Auto Topic Creation | 2.03.0 | ||||
| 68 | KIP- | 251: Allow timestamp manipulation in Processor API360: Improve handling of unknown producer | 2. | 04.0 (WIP) | ||
| 69 | KIP- | 249: Add Delegation Token Operations to KafkaAdminClient359: Verify leader epoch in produce requests | 2. | 04.0 (WIP) | ||
| 70 | KIP- | 247: Add public test utils for Kafka Streams358: Migrate Streams API to Duration instead of long ms times | 2 | 1.1.0 | ||
| 71 | KIP-245: Use Properties instead of StreamsConfig in KafkaStreams constructor357: Add support to list ACLs per principal | 2.01.0 | ||||
| 72 | KIP-244356: Add Record Header support to Kafka Streams Processor APIwithCachingDisabled() to StoreBuilder | 2.01.0 | ||||
| 73 | KIP- | 243: Make ProducerConfig and ConsumerConfig constructors public354: Add a Maximum Log Compaction Lag | 2.3 | 1.1.0 | ||
| 74 | KIP- | 239 Add queryableStoreName() to GlobalKTable353: Improve Kafka Streams Timestamp Synchronization | 2 | 1.1.0 | ||
| 75 | KIP- | 238: Expose Kafka cluster ID in Connect REST API352: Distinguish URPs caused by reassignment | 2.4.0 (WIP) | 1.1.0|||
| 76 | KIP- | 237: More Controller Health Metrics351: Add --under-min-isr option to describe topics command | 2. | 03.0 | ||
| 77 | KIP-235: Add DNS alias support for secured connection346: Improve LogCleaner behavior on error | 2.1.0 | ||||
| 78 | KIP- | 233: Simplify StreamsBuilder#addGlobalStore345: Introduce static membership protocol to reduce consumer rebalances | 2.4.0 (WIP), partially available in 2.3 | 1.1.0 | ||
| 79 | KIP-231: Improve the Required ACL of ListGroups API342 Add support for custom SASL extensions in OAuthBearer authentication | 2.1.0 | ||||
| 80 | KIP- | 229: DeleteGroups API341: Update Sticky Assignor's User Data Protocol | 2.3 | 1.1.0 | ||
| 81 | KIP | -227 - Introduce Incremental FetchRequests to Increase Partition Scalability-340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file | 2 | 1.1.0 | ||
| 82 | KIP- | 226 - Dynamic Broker Configuration339: Create a new IncrementalAlterConfigs API | 2.3.0 | 1.1.0|||
| 83 | KIP- | 225 - Use tags for consumer “records.lag” metrics338 Support to exclude the internal topics in kafka-topics.sh command | 2 | 1.1.0 | ||
| 84 | KIP- | 224: Add configuration parameter `retries` to Streams API336: Consolidate ExtendedSerializer/Serializer and ExtendedDeserializer/Deserializer | 2 | 1.1.0 | ||
| 85 | KIP-223 - Add per-topic min lead and per-partition lead metrics to KafkaConsumer332: Update AclCommand to use AdminClient API | 2.01.0 | ||||
| 86 | KIP- | 222 - Add Consumer Group operations to Admin API331 Add default implementation to close() and configure() for Serializer, Deserializer and Serde | 2. | 03.0 | ||
| 87 | KIP- | 220330: Add | AdminClient into Kafka Streams' ClientSupplierretentionPeriod in SessionBytesStoreSupplier | 2 | 1.1.0 | |
| 88 | KIP- | 219 - Improve quota communication328: Ability to suppress updates for KTables | 2.1.0 (partially implemented) / 2.3.0 (WIP) | |||
| 89 | KIP- | 218: Make KafkaFuture.Function java 8 lambda compatible324: Add method to get metrics() in AdminClient | 2 | 1.1.0 | ||
| 90 | KIP-215: Add topic regex support for Connect sinks322: Return new error code for DeleteTopics API when topic deletion disabled. | 21.1.0 | ||||
| 91 | KIP- | 214: Add zookeeper.max.in.flight.requests config to the broker321: Update TopologyDescription to better represent Source and Sink Nodes | 2 | 1.1.0 | ||
| 92 | KIP- | 212: Enforce set of legal characters for connector names320: Allow fetchers to detect and handle log truncation | 2.1.0 (partially implemented) / 2.2 | 1.1.0 | ||
| 93 | KIP- | 211: Revise Expiration Semantics of Consumer Group Offsets319: Replace segments with segmentInterval in WindowBytesStoreSupplier | 2.1.0 | |||
| 94 | KIP-313: Add KStream.flatTransform and KStream.flatTransformValues | 2.2.0 (partially implemented) / 2.3.0 | ||||
| 95 | KIP-312 Add Overloaded StreamsBuilder Build Method to Accept java.util.Properties | 2 | 210 - Provide for custom error handling when Kafka Streams fails to produce1.1.0 | 95 | KIP-208: Add SSL support to Kafka Connect REST interface | 1.1.0 |
| 96 | KIP-207: Offsets returned by ListOffsetsResponse should be monotonically increasing even during a partition leader change308: Support dynamic update of max.connections.per.ip/max.connections.per.ip.overrides | 2.12.2.0 | ||||
| 97 | KIP-206: Add support for UUID serialization and deserialization307: Allow to define custom processor names with KStreams DSL | 2.13.0 (partial) | ||||
| 98 | KIP- | 205: Add all() and range() API to ReadOnlyWindowStore306: Configuration for Delaying Response to Failed Authentication | 2 | 1.1.0 | ||
| 99 | KIP-204 : Adding records deletion operation to the new Admin Client API305: Add Connect primitive number converters | 2.01.1.0 | ||||
| 100 | KIP- | 203: Add toLowerCase support to sasl.kerberos.principal.to.local rule303: Add Dynamic Routing in Streams Sink | 2.0 | 1.1.0 | ||
| 101 | KIP | -202: Move merge() from StreamsBuilder to KStream-302 - Enable Kafka clients to use all DNS resolved IP addresses | 2. | 1.0 | .0||
| 102 | KIP- | 198: Remove ZK dependency from Streams Reset Tool300: Add Windowed KTable API in StreamsBuilder | 2.4 | 1.0.0 | ||
| 103 | KIP- | 197 Connect REST API should include the connector type when describing a connector298: Error Handling in Connect | 2 | 1.0.0 | ||
| 104 | KIP- | 196: Add metrics to Kafka Connect framework297: Externalizing Secrets for Connect Configurations | 2 | 1.0.0 | ||
| 105 | KIP- | 195: AdminClient.createPartitions295 Add Streams Configuration Allowing for Optional Topology Optimization | 2 | 1.0.0 | ||
| 106 | KIP- | 192 : Provide cleaner semantics when idempotence is enabled294 - Enable TLS hostname verification by default | 2 | 1.0.0 | ||
| 107 | 191: KafkaConsumer.subscribe() overload that takes just Pattern | Accepted1.0.0 | ||||
| 108 | KIP- | 190: Handle client-ids consistently between clients and brokers292: Add transformValues() method to KTable | 2 | 1.0.0 | ||
| 109 | KIP- | 189: Improve principal builder interface and add support for SASL290: Support for Prefixed ACLs | 2 | 1.0.0 | ||
| 110 | KIP- | 188 - Add new metrics to support health checks289: Improve the default group id behavior in KafkaConsumer | 2.2 | 1.0.0 | ||
| 111 | KIP-187 - Add cumulative count metric for all Kafka rate metrics285: Connect Rest Extension Plugin | 21.0.0 | ||||
| 112 | KIP- | 186284: | Increase offsets retention default to 7 daysSet default retention ms for Streams repartition topics to Long.MAX_VALUE | 2.0.0 | ||
| 113 | KIP- | 183 - Change PreferredReplicaLeaderElectionCommand to use AdminClient283: Efficient Memory Usage for Down-Conversion | 2.0.0 | |||
| 114 | KIP- | 182: Reduce Streams DSL overloads and allow easier use of custom storage engines282: Add the listener name to the authentication context | 2 | 1.0.0 | ||
| 115 | KIP- | 180: Add a broker metric specifying the number of consumer group rebalances in progress281: ConsumerPerformance: Increase Polling Loop Timeout and Make It Reachable by the End User | 2.0 | 1.1.0 | ||
| 116 | KIP-177: Consumer perf tool should count rebalance time279: Fix log divergence between leader and follower after fast leader fail over | 21.0.0 | ||||
| 117 | KIP-176: Remove deprecated new-consumer option for tools278 - Add version option to Kafka's commands | 2.0.0 | ||||
| 118 | KIP- | 175: Additional '--describe' views for ConsumerGroupCommand277 - Fine Grained ACL for CreateTopics API | 2.0 | 1.1.0 | ||
| 119 | KIP-174 - Deprecate and remove internal converter configs in WorkerConfig276 - Add StreamsConfig prefix for different consumers | 2.0.0 | ||||
| 120 | KIP-173: Add prefix to StreamsConfig to enable setting default internal topic configs274: Kafka Streams Skipped Records Metrics | 21.0.0 | ||||
| 121 | KIP- | 171 - Extend Consumer Group Reset Offset for Stream Application272: Add API version tag to broker's RequestsPerSec metric | 2.0 | 1.1.0 | ||
| 122 | KIP- | 168: Add GlobalTopicCount and GlobalPartitionCount metric per cluster270 - A Scala Wrapper Library for Kafka Streams | 2 | 1.0.0 | ||
| 123 | KIP-167: Add interface for the state store restoration process268: Simplify Kafka Streams Rebalance Metadata Upgrade | 21.0.0 | ||||
| 124 | KIP-164 - Add UnderMinIsrPartitionCount and per-partition UnderMinIsr metrics267: Add Processor Unit Test Support to Kafka Streams Test Utils | 21.0.0 | ||||
| 125 | KIP-163: Lower the Minimum Required ACL Permission of OffsetFetch266: Fix consumer indefinite blocking behavior | 21.0.0 | ||||
| 126 | KIP-162: Enable topic deletion by default265: Make Windowed Serde to public APIs | 21.0.0 | ||||
| 127 | KIP-161: streams deserialization exception handlers261: Add Single Value Fetch in Window Stores | 21.0.0 | ||||
| 128 | KIP- | 160: Augment KStream.print(), KStream.writeAsText() to allow users pass in extra parameters in the printed string258: Allow to Store Record Timestamps in RocksDB | 2.3.0 (partially implemented) | 1.0.0|||
| 129 | KIP- | 157 - Add consumer config options to streams reset tool257 - Configurable Quota Management | 2 | 1.0.0 | ||
| 130 | KIP-156 Add option "dry run" to Streams application reset tool255: OAuth Authentication via SASL/OAUTHBEARER | 20.11.0.0 | ||||
| 131 | KIP- | 155 - Add range scan for windowed state stores251: Allow timestamp manipulation in Processor API | 2 | 0.11.0.0 | ||
| 132 | KIP-154 Add Kafka Connect configuration properties for creating internal topics249: Add Delegation Token Operations to KafkaAdminClient | 20.11.0.0 | ||||
| 133 | KIP-153: Include only client traffic in BytesOutPerSec metric247: Add public test utils for Kafka Streams | 1.10.11.0.0 | ||||
| 134 | KIP-152 - Improve diagnostics for SASL authentication failures245: Use Properties instead of StreamsConfig in KafkaStreams constructor | 21.0.0 | ||||
| 135 | KIP- | 151 Expose Connector type in REST API244: Add Record Header support to Kafka Streams Processor API | 2 | 0.11.0.0 | ||
| 136 | KIP-150 - Kafka-Streams Cogroup243: Make ProducerConfig and ConsumerConfig constructors public | 1.1.02.2.0 (WIP) | ||||
| 137 | KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner239 Add queryableStoreName() to GlobalKTable | 1.1.0 (partially implemented) (WIP for 2.2.0) | ||||
| 138 | KIP-238: Expose Kafka cluster ID in Connect REST API | 1.1.0 | 138 | KIP-146 - Classloading Isolation in Connect | 0.11.0.0 | |
| 139 | KIP-145 - Expose Record Headers in Kafka Connect237: More Controller Health Metrics | 2.01.1.0 | ||||
| 140 | KIP- | 144: Exponential backoff for broker reconnect attempts235: Add DNS alias support for secured connection | 2.1.0 | 0.11.0.0|||
| 141 | KIP-143: Controller Health Metrics233: Simplify StreamsBuilder#addGlobalStore | 1.10.11.0.0 | ||||
| 142 | KIP- | 140: Add administrative RPCs for adding, deleting, and listing ACLs231: Improve the Required ACL of ListGroups API | 2.1.0 | 0.11.0.0|||
| 143 | KIP-138: Change punctuate semantics229: DeleteGroups API | 1.01.0 | ||||
| 144 | KIP-137: Enhance TopicCommand --describe to show topics marked for deletion227 - Introduce Incremental FetchRequests to Increase Partition Scalability | 1.10.11.0.0 | ||||
| 145 | KIP-136: Add Listener name to SelectorMetrics tags-226 - Dynamic Broker Configuration | 1.10.11.0.0 | ||||
| 146 | KIP-134: Delay initial consumer group rebalance-225 - Use tags for consumer “records.lag” metrics | 1.10.11.0.0 | ||||
| 147 | KIP-133: Describe and Alter Configs Admin APIs224: Add configuration parameter `retries` to Streams API | 1.1.00.11.0.0 | ||||
| 148 | KIP- | 130: Expose states of active tasks to KafkaStreams public API223 - Add per-topic min lead and per-partition lead metrics to KafkaConsumer | 2 | 1.0.0 | ||
| 149 | KIP-129: Streams Exactly-Once Semantics222 - Add Consumer Group operations to Admin API | 2.0.11.0.0 | ||||
| 150 | KIP-128: Add ByteArrayConverter for Kafka Connect221: Enhance DSL with Connecting Topic Creation and Repartition Hint | 2.4.0 (WIP)0.11.0.0 | ||||
| 151 | KIP-126 - Allow KafkaProducer to split and resend oversized batches.220: Add AdminClient into Kafka Streams' ClientSupplier | 1.1.00.11.0.0 | ||||
| 152 | KIP-124 - Request rate quotas219 - Improve quota communication | 20.11.0.0 | ||||
| 153 | KIP-123: Allow per stream/table timestamp extractor218: Make KafkaFuture.Function java 8 lambda compatible | 1.10.11.0.0 | ||||
| 154 | KIP-122215: Add Reset Consumer Group Offsets toolingtopic regex support for Connect sinks | 1.10.11.0.0 | ||||
| 155 | KIP-121214: Add KStream peek methodzookeeper.max.in.flight.requests config to the broker | 1.1.00.11.0.0 | ||||
| 156 | KIP- | 120: Cleanup Kafka Streams builder API213 Support non-key joining in KTable | 2.4.0 (WIP) | 1.0.0|||
| 157 | KIP-119: Drop Support for Scala 2.10 in Kafka 0212: Enforce set of legal characters for connector names | 1.1.110.11.0.0 | ||||
| 158 | KIP-118: Drop Support for Java 7211: Revise Expiration Semantics of Consumer Group Offsets | 2.01.0 | ||||
| 159 | KIP-117: Add a public AdminClient API for Kafka admin operations-210 - Provide for custom error handling when Kafka Streams fails to produce | 1.10.11.0.0 | ||||
| 160 | KIP-208: Add SSL support to Kafka Connect REST interface | 1.1115: Enforce offsets.topic.replication.factor upon __consumer_offsets auto topic creation0.11.0.0 | ||||
| 161 | KIP-114: KTable state stores and improved semantics207: Offsets returned by ListOffsetsResponse should be monotonically increasing even during a partition leader change | 2.20.11.0.0 | ||||
| 162 | KIP- | 113: Support replicas movement between log directories206: Add support for UUID serialization and deserialization | 2 | 1.1.0 | ||
| 163 | KIP-112: Handle disk failure for JBOD205: Add all() and range() API to ReadOnlyWindowStore | 1.01.0 | ||||
| 164 | KIP- | 110: Add Codec for ZStandard Compression204 : Adding records deletion operation to the new Admin Client API | 1 | 2.1.0 | ||
| 165 | KIP-109: Old Consumer Deprecation203: Add toLowerCase support to sasl.kerberos.principal.to.local rule | 1.10.11.0.0 | ||||
| 166 | KIP-108: Create Topic Policy202: Move merge() from StreamsBuilder to KStream | 1.0.0.10.2.0 | ||||
| 167 | KIP-107: Add deleteRecordsBefore() API in AdminClient198: Remove ZK dependency from Streams Reset Tool | 10.11.0.0 | ||||
| 168 | KIP-197 Connect REST API should include the connector type when describing a connector | 1106 - Change Default unclean.leader.election.enabled from True to False0.11.0.0 | ||||
| 169 | KIP-105: Addition of Record Level for Sensors196: Add metrics to Kafka Connect framework | 1.0.0.10.2.0 | ||||
| 170 | KIP-104: Granular Sensors for Streams 195: AdminClient.createPartitions | 1.0.0.10.2.0 | ||||
| 171 | KIP-103: Separation of Internal and External traffic192 : Provide cleaner semantics when idempotence is enabled | 1.00.10.2.0 | ||||
| 172 | KIP-102 - Add close with timeout for consumers191: KafkaConsumer.subscribe() overload that takes just Pattern | 1.0.0.10.2.0 | ||||
| 173 | KIP-101 - Alter Replication Protocol to use Leader Epoch rather than High Watermark for Truncation190: Handle client-ids consistently between clients and brokers | 10.11.0.0 | ||||
| 174 | KIP-100 - Relax Type constraints in Kafka Streams API189: Improve principal builder interface and add support for SASL | 1.0.0.10.2.0 | ||||
| 175 | KIP-188 - 99: Add Global Tables to Kafka Streamsnew metrics to support health checks | 1.0.0.10.2.0 | ||||
| 176 | KIP-98 - Exactly Once Delivery and Transactional Messaging187 - Add cumulative count metric for all Kafka rate metrics | 1.0.11.0.0 | ||||
| 177 | KIP-97: Improved Kafka Client RPC Compatibility Policy186: Increase offsets retention default to 7 days | 2.0.0.10.2.0 | ||||
| 178 | KIP-96 - Add per partition metrics for in-sync and assigned replica count183 - Change PreferredReplicaLeaderElectionCommand to use AdminClient | 20.10.2.0 | ||||
| 179 | KIP-94 Session Windows182: Reduce Streams DSL overloads and allow easier use of custom storage engines | 1.00.10.2.0 | ||||
| 180 | KIP-93: Improve invalid timestamp handling in Kafka Streams180: Add a broker metric specifying the number of consumer group rebalances in progress | 1.10.10.2.0 | ||||
| 181 | KIP-92 - Add per partition lag metrics to KafkaConsumer177: Consumer perf tool should count rebalance time | 1.00.10.2.0 | ||||
| 182 | KIP- | 91 Provide Intuitive User Timeouts in The Producer176: Remove deprecated new-consumer option for tools | 2. | 10.0 | ||
| 183 | KIP-90 - Remove zkClient dependency from Streams175: Additional '--describe' views for ConsumerGroupCommand | 1.10.10.2.0 | ||||
| 184 | KIP-89: Allow sink connectors to decouple flush and offset commit174 - Deprecate and remove internal converter configs in WorkerConfig | 2.0.0.10.2.0 | ||||
| 185 | KIP-88: OffsetFetch Protocol Update173: Add prefix to StreamsConfig to enable setting default internal topic configs | 1.00.10.2.0 | ||||
| 186 | KIP-86: Configurable SASL callback handlers-171 - Extend Consumer Group Reset Offset for Stream Application | 1.12.0.0 | ||||
| 187 | KIP-85: Dynamic JAAS configuration for Kafka clients168: Add GlobalTopicCount and GlobalPartitionCount metric per cluster | 1.0.0.10.2.0 | ||||
| 188 | KIP-84: Support SASL SCRAM mechanisms167: Add interface for the state store restoration process | 1.00.10.2.0 | ||||
| 189 | 82 Record Headers UnderMinIsrPartitionCount and per-partition UnderMinIsr metrics | 1.0.11.0.0 | ||||
| 190 | 81: Bound Fetch memory usage in the consumer | 1.0.0 | 2.2.0 (WIP)||||
| 191 | KIP-79 - ListOffsetRequest/ListOffsetResponse v1 and add timestamp search methods to the new consumer162: Enable topic deletion by default | 1.0.0.10.1.0 | ||||
| 192 | KIP-78: Cluster Id161: streams deserialization exception handlers | 1.00.10.1.0 | ||||
| 193 | KIP-77: Improve Kafka Streams Join Semantics160: Augment KStream.print(), KStream.writeAsText() to allow users pass in extra parameters in the printed string | 1.0.0.10.2.0 | ||||
| 194 | KIP- | 75157 - Add | per-connector Converters0.10.1.0consumer config options to streams reset tool | 1.0.0 | ||
| 195 | KIP- | 74: Add Fetch Response Size Limit in Bytes156 Add option "dry run" to Streams application reset tool | 0. | 1011. | 10.0 | |
| 196 | KIP- | 73: Replication Quotas155 - Add range scan for windowed state stores | 0. | 1011. | 10.0 | |
| 197 | KIP-72: Allow putting a bound on memory consumed by Incoming request 154 Add Kafka Connect configuration properties for creating internal topics | 0.111.0.0 | ||||
| 198 | KIP-71: Enable log compaction and deletion to co-exist153: Include only client traffic in BytesOutPerSec metric | 0.1011.10.0 | ||||
| 199 | KIP-70: Revise Partition Assignment Semantics on New Consumer's Subscription Change-152 - Improve diagnostics for SASL authentication failures | 1.0.0.10.1.0 | ||||
| 200 | KIP-67: Queryable state for Kafka Streams151 Expose Connector type in REST API | 0.1011.10.0 | ||||
| 201 | KIP-66: Single Message Transforms for Kafka Connect150 - Kafka-Streams Cogroup | 2.4.0 (WIP)0.10.2.0 / 0.11.0.0 | ||||
| 202 | KIP-65: Expose timestamps to Connect149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner | 1.1.0 (partially implemented) (WIP for 2.4.0)0.10.1.0 | ||||
| 203 | KIP-63: Unify store and downstream caching in streams146 - Classloading Isolation in Connect | 0. 1011. 10.0 | ||||
| 204 | KIP-62: Allow consumer to send heartbeats from a background thread-145 - Expose Record Headers in Kafka Connect | 1.0.10.1.0 | ||||
| 205 | KIP-60 - Make Java client classloading more flexible144: Exponential backoff for broker reconnect attempts | 0.1011.10.0 | ||||
| 206 | KIP-58 - Make Log Compaction Point Configurable143: Controller Health Metrics | 0.1011.10.0 | ||||
| 207 | KIP-57 - Interoperable LZ4 Framing140: Add administrative RPCs for adding, deleting, and listing ACLs | 0.1011.0.0 | ||||
| 208 | KIP-56: Allow cross origin HTTP requests on all HTTP methods138: Change punctuate semantics | 10.10.0.0 | ||||
| 209 | KIP-55: Secure Quotas for Authenticated Users137: Enhance TopicCommand --describe to show topics marked for deletion | 0.1011.10.0 | ||||
| 210 | KIP-54: Sticky Partition Assignment Strategy136: Add Listener name to SelectorMetrics tags | 0.11.0.0 | ||||
| 211 | KIP-52: Connector Control APIs134: Delay initial consumer group rebalance | 0.1011.0.0 | ||||
| 212 | KIP-51 - List Connectors REST API133: Describe and Alter Configs Admin APIs | 0.1011.0.0 | ||||
| 213 | KIP- | 50 - Move Authorizer to o.a.k.common package0.10.1.0 | ||||
| 214 | KIP-48 Delegation token support for Kafka | 1.1.0 | ||||
| 130: Expose states of active tasks to KafkaStreams public API | 1.0.0 | |||||
| 214 | KIP-129: Streams Exactly-Once Semantics | 0.11 | 215 | KIP-45 - Standardize all client sequence interaction on j.u.Collection. | 0.10.0.0 | |
| 216215 | KIP-43128: Add ByteArrayConverter for Kafka SASL enhancementsConnect | 0.1011.0.0 | ||||
| 217216 | KIP-42: Add Producer and Consumer Interceptors126 - Allow KafkaProducer to split and resend oversized batches. | 0.11.0.0 | ||||
| 217 | KIP-124 - Request rate quotas | 0.1011.0.0 | ||||
| 218 | KIP-41: Consumer Max Records123: Allow per stream/table timestamp extractor | 0.1011.0.0 | ||||
| 219 | KIP-40: ListGroups and DescribeGroup122: Add Reset Consumer Group Offsets tooling | 0.911.0.0 | ||||
| 220 | KIP-38: ZooKeeper Authentication121: Add KStream peek method | 0. 911.0.0 | ||||
| 221 | KIP-36 - Rack aware replica assignment120: Cleanup Kafka Streams builder API | 10.10.0.0 | ||||
| 222 | KIP-35 - Retrieving protocol version119: Drop Support for Scala 2.10 in Kafka 0.11 | 0.1011.0.0 | ||||
| 223 | KIP-33 - Add a time based log index118: Drop Support for Java 7 | 2.0.0.10.1.0 | ||||
| 224 | KIP-32 - Add timestamps to Kafka message117: Add a public AdminClient API for Kafka admin operations | 0.1011.0.0 | ||||
| 225 | KIP-31 - Move to relative offsets in compressed message sets115: Enforce offsets.topic.replication.factor upon __consumer_offsets auto topic creation | 0.1011.0.0 | ||||
| 226 | KIP-28 - Add a processor client114: KTable state stores and improved semantics | 0.1011.0.0 | ||||
| 227 | KIP-26 - Add Kafka Connect framework for data import/export113: Support replicas movement between log directories | 1.10.9.0.0 | ||||
| 228 | KIP-25 - System test improvements112: Handle disk failure for JBOD | 10.9.0.0 | ||||
| 229 | KIP- | 22 - Expose a Partitioner interface in the new producer110: Add Codec for ZStandard Compression | 2.1.0 | 0.9.0.0|||
| 230 | KIP-21 - Dynamic Configuration109: Old Consumer Deprecation | 0.911.0.0 (WIP) | ||||
| 231 | KIP-20 Enable log preallocate to improve consume performance under windows and some old Linux file system108: Create Topic Policy | 0.910.02.0 | ||||
| 232 | KIP-19 - Add a request timeout to NetworkClient107: Add deleteRecordsBefore() API in AdminClient | 0.911.0.0 | ||||
| 233 | 16 - Automated Replica Lag Tuning 106 - Change Default unclean.leader.election.enabled from True to False | 0.911.0.0 | ||||
| 234 | KIP-15 - Add a close method with a timeout in the producer105: Addition of Record Level for Sensors | 0.910.02.0 | ||||
| 235 | KIP-13 - Quota Design104: Granular Sensors for Streams | 0.910.02.0 | ||||
| 236 | KIP-12 - Kafka Sasl/Kerberos and SSL implementation103: Separation of Internal and External traffic | 0.910.02.0 | ||||
| 237 | KIP-11 - Kafka Authorizer design102 - Add close with timeout for consumers | 0.910.02.0 | ||||
| 238 | KIP-8 - Add a flush method to the producer API101 - Alter Replication Protocol to use Leader Epoch rather than High Watermark for Truncation | 0.911.0.0 | ||||
| 239 | KIP-4 - Metadata Protocol Changes100 - Relax Type constraints in Kafka Streams API | 0.10.02.0 | ||||
| 240 | KIP-4 - Command line and centralized administrative operations99: Add Global Tables to Kafka Streams | 0.10.2.00.9.0.0, 0.10.0.0, 0.10.1.0 | ||||
| 241 | KIP-3 - Mirror Maker Enhancement98 - Exactly Once Delivery and Transactional Messaging | 0.911.0.0 | ||||
| 242 | KIP-2 - Refactor brokers to allow listening on multiple ports and IPs97: Improved Kafka Client RPC Compatibility Policy | 0.910.02.0 | ||||
| 243 | KIP-1 - Remove support of request.required.acks-96 - Add per partition metrics for in-sync and assigned replica count | 0.910.02.0 |
...
...
KIPs under discussion
Please insert new rows in sorted order (ascending by KIP number). Please move the adopted/accepted KIPs to the "Adopted KIPs" table above.
Dormant/inactive KIPs
Please insert new rows in sorted order (ascending by KIP number).
Discarded KIPs
Please insert new rows in sorted order (ascending by KIP number).
...
Discarded KIPs
| KIP | Comment | |
|---|---|---|
| KIP-5 - Broker Configuration Management | Superseded by KIP-21 | |
| KIP-7 - Security - IP Filtering | ||
| KIP-9 - SSL Support | Adopted via KAFKA-1690 but not via this KIP process | |
| KIP-18 - JBOD Support | Superseded by KIP-112 and KIP-113 | |
| KIP-24 - Remove ISR information from TopicMetadataRequest and add broker level metadata request | ||
| KIP-29 - Add an IsrPropagateIntervalMs configuration to KafkaConfig | No longer need after KAFKA-2722 | |
KIP-34 Add Partitioner Change Listener to Partitioner Interface for Multiple Use Case | Per feedback it is better do this parition() method and avoid Thread Coordination etc. | |
| KIP-80: Kafka Rest Server | ||
| KIP-83 - Allow multiple SASL authenticated Java clients in a single JVM process | After KIP-85, no interface changes, reduced to KAFKA-4180 | |
| KIP-116: Add State Store Checkpoint Interval Configuration | Currently not needed as checkpointing can be done on commit interval. | |
| KIP-127: Pluggable JAAS LoginModule configuration for SSL | Similar functionality can be implemented using a custom PrincipalBuilder. | |
| KIP-132: Augment KStream.print to allow extra parameters in the printed string | Duplicated by KIP-160 | |
| KIP-184: Rename LogCleaner and related classes to LogCompactor | A lot of configuration change will cause more trouble and doesn't seems to be worth it | |
| KIP-111: Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker. | Covered by KIP-189 | |
| KIP-147: Add missing type parameters to StateStoreSupplier factories and KGroupedStream/Table methods | Covered by KIP-182 | |
| KIP-170: Enhanced TopicCreatePolicy and introduction of TopicDeletePolicy | Superseded by KIP-201: Rationalising Policy interfaces | |
| KIP 172: Add regular-expression topic support for sink connector | Superseded by KIP-215: Add topic regex support for Connect sinks | KIP-179: Change ReassignPartitionsCommand to use AdminClient |
| KIP 230: Name Windowing Joins | Covered by KIP-372: Naming Repartition Topics for Joins and Grouping | |
| KIP-232: Detect outdated metadata using per-partition leaderEpoch field | Merged into KIP-320: Allow fetchers to detect and handle log truncation | |
| KIP-246: Connect producers and consumers should inherit worker configs | Withdrawn because proposal could not be made backward compatible with existing behavior | |
| KIP-263: Allow broker to skip sanity check of inactive segments on broker startup | Withdrawn because the solution we agreed on does not require interface change | |
| KIP-286: producer.send() should not block on metadata update | Withdrawn because the benefit of not having to wait for metadata is probably not worth the complexity added in producer. | |
| KIP-288: [DISCARDED] Consumer.poll() timeout semantic change and new waitForAssignment method | Discarded in deference to KIP-266 | |
| KIP-310: Add a Kafka Source Connector to Kafka Connect | Withdrawn in favor of KIP-382. | |
| KIP-327: Add describe all topics API to AdminClient | Withdrawn in-favor of filter support in Metadata API and KIP-142 | |
| KIP-344: The auto-generated client id should be passed to MetricsReporter | Bug fix so KIP not needed | |
| KIP-364: Remove implicit Materialized, Consumed and Produced | Subsumed by KIP-365 | |
| KIP-404: Add Kafka Connect configuration parameter for disabling WADL output on OPTIONS request | Discarded, reported as a bug by | Jira | |
| server | ASF JIRA | |
| serverId | 5aa69414-a9e9-3523-82ec-879b028fb15b | key | KAFKA-7759
KIP Discussion Recordings
...