This page describes a proposed Kafka Improvement Proposal (KIP) process for proposing a major change to Kafka.
To create your own KIP, click on
| Create from template | ||||||||
|---|---|---|---|---|---|---|---|---|
|
dev@kafka.apache.org and request permission (http://kafka.apache.org/contact). Also add an entry to the table KIPs under discussion (for Streams API KIPs, please also add it to Kafka Streams sub page)....
- Click
. Take the next available KIP number and give your proposal a descriptive heading. e.g. "KIP 42: Allow Infinite Retention With Bounded Disk Usage".Create from template templateName 54329345 templateId 54329345 title KIP-NEXT: Insert Title Here buttonLabel Create KIP - Fill in the sections as described above
- Start a [DISCUSS] thread on the Apache mailing list. Please ensure that the subject of the thread is of the format [DISCUSS] KIP-{your KIP number} {your KIP heading} The and the body contains a link to your new KIP. The discussion should happen on the mailing list, not on the wiki, since the wiki comment system doesn't work well for larger discussions. In the process of the discussion you may update the proposal. You should let people know the changes you are making. When you feel you have a finalized proposal
- Once the proposal is finalized call a [VOTE] to have the proposal adopted. These proposals are more serious than code changes and more serious even than release votes. The criteria for acceptance is lazy majority. The vote should remain open for at least 72 hours.
- Please update the KIP wiki page, and the index below, to reflect the current stage of the KIP after a vote. This acts as the permanent record indicating the result of the KIP (e.g., Accepted or Rejected). Also report the result of the KIP vote to the voting thread on the mailing list so the conclusion is clear.
KIP round-up
Next KIP Number: 373541
Use this number as the identifier for your KIP and increment this value.
Adopted KIPs
Please insert new rows in sorted order (descending by KIP number).
| KIP (please keep this sorted by KIP number) | Release | |||||
|---|---|---|---|---|---|---|
| 1 | KIP- | 366: Make FunctionConversions deprecated2.1.0534: Reorganize checkpoint system in log cleaner to per partition | WIP | |||
| 2 | 365: Materialized, Serialized, Joined, Consumed and Produced with implicit Serde | WIP2.1.0 | ||||
| 3 | KIP-359: Verify leader epoch in produce requests531: Drop support for Scala 2.11 in Kafka 2.5 | 2.15.0 (WIP) | ||||
| 4 | KIP- | 356: Add withCachingDisabled() to StoreBuilder528: Deprecate PartitionGrouper configuration and interface | 2. | 14.0 | ||
| 5 | KIP- | 353: Improve Kafka Streams Timestamp Synchronization525 - Return topic metadata and configs in CreateTopics response | 2. | 14. | 00 | |
| 6 | KIP-342 Add support for custom SASL extensions in OAuthBearer authentication524: Allow users to choose config source when describing configs | 2.15.0 (WIP) | ||||
| 7 | KIP- | 341: Update Sticky Assignor's User Data Protocol521: Enable redirection of Connect's log4j messages to a file by default | 2. | 14.0 | (WIP)||
| 8 | KIP-338 Support to exclude the internal topics in kafka-topics.sh command517: Add consumer metrics to observe user poll behavior | 2.14.0 | ||||
| 9 | KIP-340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file | 2.1.0 | 10 | KIP-336: Consolidate ExtendedSerializer/Serializer and ExtendedDeserializer/Deserializer | 511: Collect and Expose Client's Name and Version in the Brokers | 2.4.0 (WIP)/2.52.1.0(WIP) |
| 1110 | KIP-332: Update AclCommand to use AdminClient API507: Securing Internal Connect REST Endpoints | 2.14.0 | ||||
| 11 | KIP-330: Add retentionPeriod in SessionBytesStoreSupplier504 - Add new Java Authorizer Interface | 2.14.00 | ||||
| 12 | KIP-328: Ability to suppress updates for KTables503: Add metric for number of topics marked for deletion | 2.14.00 | ||||
| 1413 | KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum | |||||
| 14 | KIP-497: Add inter-broker API to alter ISR | 2.5.0 (WIP)324: Add method to get metrics() in AdminClient2.1.0 | ||||
| 15 | KIP- | 322: Return new error code for DeleteTopics API when topic deletion disabled.496: Administrative API to delete consumer offsets | 2. | 14. | 00 | |
| 16 | KIP- | 321: Update TopologyDescription to better represent Source and Sink Nodes495: Dynamically Adjust Log Levels in Connect | 2. | 14.0 | ||
| 17 | KIP-320: Allow fetchers to detect and handle log truncation492: Add java security providers in Kafka Security config | 2.14.00 | ||||
| 18 | KIP-319: Replace segments with segmentInterval in WindowBytesStoreSupplier488: Clean up Sum,Count,Total Metrics | 2.14.0 | ||||
| 19 | KIP- | 308: Support dynamic update of max.connections.per.ip/max.connections.per.ip.overrides484: Expose metrics for group and transaction metadata loading duration | 2. | 14. | 00 | |
| 20 | KIP-306: Configuration for Delaying Response to Failed Authentication482: The Kafka Protocol should Support Optional Tagged Fields | 2.14.00 | ||||
| 21 | KIP- | 305481: | Add Connect primitive number convertersSerDe Improvements for Connect Decimal type in JSON | 2. | 04.0 | |
| 22 | KIP- | 303: Add Dynamic Routing in Streams Sink480: Sticky Partitioner | 2. | 04.0 | ||
| 23 | KIP- | 298: Error Handling in Connect479: Add Materialized to Join | 2. | 04.0 | ||
| 24 | KIP-297: Externalizing Secrets for Connect Configurations476: Add Java AdminClient Interface | 2.04.0 | ||||
| 25 | KIP- | 295 Add Streams Configuration Allowing for Optional Topology Optimization475: New Metrics to Measure Number of Tasks on a Connector | 2. | 04.0 | ||
| 26 | KIP- | 294 - Enable TLS hostname verification by default474: To deprecate WindowStore#put(key, value) | 2. | 04.0 | ||
| 27 | KIP- | 292: Add transformValues() method to KTable471: Expose RocksDB Metrics in Kafka Streams | 2.4.0 (partially implemented) / 2.5 | 2.0.0 | ||
| 28 | KIP-290: Support for Prefixed ACLs470: TopologyTestDriver test input and output usability improvements | 2.04.00 | ||||
| 29 | KIP-289: Improve the default group id behavior in KafkaConsumer467: Augment ProduceResponse error messaging for specific culprit records | 2.14.0 (WIP)partially implemented) / 2.5.0 | ||||
| 30 | KIP- | 285: Connect Rest Extension Plugin466: Add support for List<T> serialization and deserialization | 2. | 04.0 (WIP) | ||
| 31 | KIP-284: Set default retention ms for Streams repartition topics to Long.MAX_VALUE465: Add Consolidated Connector Endpoint to Connect REST API | 2.03.0 | ||||
| 32 | KIP-283464: Efficient Memory Usage for Down-ConversionDefaults for AdminClient#createTopic | 2.04.0 | ||||
| 33 | KIP-282: Add the listener name to the authentication context462: Use local thread id for KStreams | 2.03.0 | ||||
| 34 | KIP-281: ConsumerPerformance: Increase Polling Loop Timeout and Make It Reachable by the End User461: Improve Replica Fetcher behavior at handling partition failure | 2.03.0 | ||||
| 35 | KIP-279: Fix log divergence between leader and follower after fast leader fail over460: Admin Leader Election RPC | 2.04.0 | ||||
| 36 | KIP- | 278 - Add version option to Kafka's commands458: Connector Client Config Override Policy | 2. | 03.0 | ||
| 37 | KIP-277 - Fine Grained ACL for CreateTopics API455: Create an Administrative API for Replica Reassignment | 2.04.0 (WIP) | ||||
| 38 | KIP- | 276 - Add StreamsConfig prefix for different consumers454: Expansion of the ConnectClusterState interface | 2. | 03.0 | ||
| 39 | KIP- | 274: Kafka Streams Skipped Records Metrics453: Add close() method to RocksDBConfigSetter | 2. | 03.0 | ||
| 40 | KIP- | 272449: Add | API version tag to broker's RequestsPerSec metricconnector contexts to log messages in Connect workers | 2. | 03.0 | |
| 41 | KIP-270 - A Scala Wrapper Library for Kafka Streams446: Add changelog topic configuration to KTable suppress | 2.05.0 (WIP) | ||||
| 42 | KIP- | 268: Simplify Kafka Streams Rebalance Metadata Upgrade445: In-memory Session Store | 2. | 03.0 | ||
| 43 | KIP- | 267: Add Processor Unit Test Support to444: Augment metrics for Kafka Streams | Test Utils2. | 04.0 (WIP) | ||
| 44 | 266: Fix consumer indefinite blocking behavior | 2.03.0 | ||||
| 45 | KIP- | 265442: | Make Windowed Serde to public APIsReturn to default max poll interval in Streams | 2. | 03.0 | |
| 46 | KIP- | 261: Add Single Value Fetch in Window Stores440: Extend Connect Converter to support headers | 2. | 04.0 | ||
| 47 | KIP- | 257 - Configurable Quota Management436: Add a metric indicating start time | 2. | 03.0 | ||
| 48 | KIP-255: OAuth Authentication via SASL/OAUTHBEARER434: Add Replica Fetcher and Log Cleaner Count Metrics | 2.04.0 (WIP) | ||||
| 49 | KIP | -251: Allow timestamp manipulation in Processor API-430 - Return Authorized Operations in Describe Responses | 2. | 03.0 | ||
| 50 | KIP- | 249: Add Delegation Token Operations to KafkaAdminClient429: Kafka Consumer Incremental Rebalance Protocol | 2. | 04.0 (WIP) | ||
| 51 | KIP-247428: Add public test utils for Kafka Streamsin-memory window store | 2.31.1.0 | ||||
| 52 | KIP-245: Use Properties instead of StreamsConfig in KafkaStreams constructor427: Add AtMinIsr topic partition category (new metric & TopicCommand option) | 2.03.0 | ||||
| 53 | KIP-244425: Add Record Header support to Kafka Streams Processor APIsome Log4J Kafka Appender Properties for Producing to Secured Brokers | 2.03.0 | ||||
| 54 | KIP- | 243: Make ProducerConfig and ConsumerConfig constructors public421: Support resolving externalized secrets in AbstractConfig | 2.3 | 1.1.0 | ||
| 55 | KIP-239 Add queryableStoreName() to GlobalKTable420: Add Single Value Fetch in Session Stores | 2.21.1.0 | ||||
| 56 | KIP-238: Expose Kafka cluster ID in Connect REST API417: Allow JmxTool to connect to a secured RMI port | 2.31.1.0 | ||||
| 57 | KIP-237: More Controller Health Metrics415: Incremental Cooperative Rebalancing in Kafka Connect | 2.03.0 | ||||
| 58 | KIP- | 233: Simplify StreamsBuilder#addGlobalStore414: Expose Embedded ClientIds in Kafka Streams | 2.2 | 1.1.0 | ||
| 59 | KIP- | 231: Improve the Required ACL of ListGroups API412: Extend Admin API to support dynamic application log levels | 2. | 1.0 (WIP)4.0 | ||
| 60 | 229: DeleteGroups API | 2.31.1.0 | ||||
| 61 | KIP- | 227 - Introduce Incremental FetchRequests to Increase Partition Scalability402: Improve fairness in SocketServer processors | 2.2.0 (partially implemented) / 2.3 | 1.1.0 | ||
| 62 | KIP- | 226 - Dynamic Broker Configuration401: TransformerSupplier/ProcessorSupplier StateStore connecting | 2.5.0 (WIP) | 1.1.0|||
| 63 | KIP- | 225 - Use tags for consumer “records.lag” metrics396: Add Commit/List Offsets Operations to AdminClient | 2.5.0 (WIP) | 1.1.0|||
| 64 | KIP- | 224: Add configuration parameter `retries` to Streams API394: Require member.id for initial join group request | 2.2 | 1.1.0 | ||
| 65 | KIP- | 223 - Add per-topic min lead and per-partition lead metrics to KafkaConsumer393: Time windowed serde to properly deserialize changelog input topic | 2. | 02.0 | ||
| 66 | KIP- | 222 - Add Consumer Group operations to Admin API392: Allow consumers to fetch from closest replica | 2. | 04.0 | ||
| 67 | KIP-220: Add AdminClient into Kafka Streams' ClientSupplier389: Introduce a configurable consumer group size limit | 2.21.1.0 | ||||
| 68 | KIP- | 219 - Improve quota communication386: Standardize on Min/Avg/Max metrics' default value | 2. | 02.0 | ||
| 69 | KIP- | 218382: | Make KafkaFuture.Function java 8 lambda compatibleMirrorMaker 2.0 | 2.4 | 1.1.0 | |
| 70 | 215: Add topic regex support for Connect sinks | 2.4.0 1.1.0 | ||||
| 71 | KIP- | 214: Add zookeeper.max.in.flight.requests config to the broker380: Detect outdated control requests and bounced brokers using broker generation | 2.2 | 1.1.0 | ||
| 72 | KIP- | 212: Enforce set of legal characters for connector names377: TopicCommand to use AdminClient | 2.2 | 1.1.0 | ||
| 73 | KIP- | 211: Revise Expiration Semantics of Consumer Group Offsets376: Implement AutoClosable on appropriate classes that want to be used in a try-with-resource statement | 2. | 1.0 (WIP)2.0 | ||
| 74 | KIP-374: Add '- | 210 - Provide for custom error handling when Kafka Streams fails to produce-help' option to all available Kafka CLI commands | 2.2 | 1.1.0 | ||
| 75 | KIP- | 208: Add SSL support to Kafka Connect REST interface372: Naming Repartition Topics for Joins and Grouping | 2 | 1.1.0 | ||
| 76 | KIP-207: Offsets returned by ListOffsetsResponse should be monotonically increasing even during a partition leader change371: Add a configuration to build custom SSL principal name | 2.02.0 (WIP) | ||||
| 77 | KIP- | 206: Add support for UUID serialization and deserialization369: Alternative Partitioner to Support "Always Round-Robin" Selection | 2. | 14.0 | ||
| 78 | KIP -205: Add all() and range() API to ReadOnlyWindowStore368: Allow SASL Connections to Periodically Re-Authenticate | 2.21.1.0 | ||||
| 79 | KIP- | 204 : Adding records deletion operation to the new Admin Client API367 Introduce close(Duration) to Producer and AdminClient instead of close(long, TimeUnit) | 2.2 | 1.1.0 | ||
| 80 | KIP-203: Add toLowerCase support to sasl.kerberos.principal.to.local rule 366: Make FunctionConversions deprecated | 21.1.0 | ||||
| 81 | KIP- | 202: Move merge() from StreamsBuilder to KStream365: Materialized, Serialized, Joined, Consumed and Produced with implicit Serde | 2. | 1.0 | .0||
| 82 | KIPKIP- | 198: Remove ZK dependency from Streams Reset Tool361: Add Consumer Configuration to Disable Auto Topic Creation | 2.3 | 1.0.0 | ||
| 83 | KIP- | 197 Connect REST API should include the connector type when describing a connector360: Improve handling of unknown producer | 2.4.0 (WIP) | 1.0.0|||
| 84 | KIP- | 196: Add metrics to Kafka Connect framework359: Verify leader epoch in produce requests | 2.4.0 (WIP) | 1.0.0|||
| 85 | KIP- | 195: AdminClient.createPartitions358: Migrate Streams API to Duration instead of long ms times | 2.1 | 1.0.0 | ||
| 86 | KIP-192 : Provide cleaner semantics when idempotence is enabled357: Add support to list ACLs per principal | 2.1.0.0 | ||||
| 87 | KIP- | 191: KafkaConsumer.subscribe() overload that takes just Pattern356: Add withCachingDisabled() to StoreBuilder | 2.1 | 1.0.0 | ||
| 88 | KIP- | 190: Handle client-ids consistently between clients and brokers354: Add a Maximum Log Compaction Lag | 2.3 | 1.0.0 | ||
| 89 | KIP- | 189353: Improve | principal builder interface and add support for SASLKafka Streams Timestamp Synchronization | 2. | 1.0 | .0|
| 90 | KIP- | 188 - Add new metrics to support health checks352: Distinguish URPs caused by reassignment | 2.5.0 | 1.0.0|||
| 91 | KIP | -187 - Add cumulative count metric for all Kafka rate metrics-351: Add --under-min-isr option to describe topics command | 2.3 | 1.0.0 | ||
| 92 | KIP- | 186: Increase offsets retention default to 7 days346: Improve LogCleaner behavior on error | 2. | 0.01 | ||
| 93 | KIP- | 183 - Change PreferredReplicaLeaderElectionCommand to use AdminClient345: Introduce static membership protocol to reduce consumer rebalances | 2.4.0 (WIP), partially available in 2.3 | 2.0.0 | ||
| 94 | KIP- | 182: Reduce Streams DSL overloads and allow easier use of custom storage engines342 Add support for custom SASL extensions in OAuthBearer authentication | 2.1 | 1.0.0 | ||
| 95 | KIP- | 180: Add a broker metric specifying the number of consumer group rebalances in progress341: Update Sticky Assignor's User Data Protocol | 2.3 | 1.1.0 | ||
| 96 | KIP- | 177: Consumer perf tool should count rebalance time340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file | 2. | 1.0 | .0||
| 97 | KIP-176339: Remove deprecated new-consumer option for toolsCreate a new IncrementalAlterConfigs API | 2.03.00 | ||||
| 98 | KIP- | 175: Additional '--describe' views for ConsumerGroupCommand338 Support to exclude the internal topics in kafka-topics.sh command | 2 | 1.1.0 | ||
| 99 | KIP- | 174 - Deprecate and remove internal converter configs in WorkerConfig336: Consolidate ExtendedSerializer/Serializer and ExtendedDeserializer/Deserializer | 2. | 01.0 | ||
| 100 | KIP-173332: Add prefix to StreamsConfig to enable setting default internal topic configsUpdate AclCommand to use AdminClient API | 2.11.0.0 | ||||
| 101 | KIP- | 171 - Extend Consumer Group Reset Offset for Stream Application331 Add default implementation to close() and configure() for Serializer, Deserializer and Serde | 2.3 | 1.1.0 | ||
| 102 | KIP- | 168330: Add | GlobalTopicCount and GlobalPartitionCount metric per clusterretentionPeriod in SessionBytesStoreSupplier | 2.1 | 1.0.0 | |
| 103 | KIP- | 167: Add interface for the state store restoration process328: Ability to suppress updates for KTables | 2.1.0 (partially implemented) / 2.3.0 (WIP) | 1.0.0|||
| 104 | KIP- | 164 - Add UnderMinIsrPartitionCount and per-partition UnderMinIsr metrics324: Add method to get metrics() in AdminClient | 2. | 1.0 | .0||
| 105 | KIP- | 163: Lower the Minimum Required ACL Permission of OffsetFetch322: Return new error code for DeleteTopics API when topic deletion disabled. | 2.1 | 1.0.0 | ||
| 106 | KIP- | 162: Enable topic deletion by default321: Update TopologyDescription to better represent Source and Sink Nodes | 2.1 | 1.0.0 | ||
| 107 | KIP- | 161: streams deserialization exception handlers320: Allow fetchers to detect and handle log truncation | 2. | 1.0 (partially implemented) / 2.2.0 | ||
| 108 | KIP-319: Replace segments with segmentInterval in WindowBytesStoreSupplier | 2.1.0 | ||||
| 109 | KIP-313: Add KStream.flatTransform and KStream.flatTransformValues | 2.2.0 (partially implemented) / 2.3 | 160: Augment KStream.print(), KStream.writeAsText() to allow users pass in extra parameters in the printed string1.0.0 | 109 | KIP-157 - Add consumer config options to streams reset tool | 1.0 .0 |
| 110 | KIP-156 Add option "dry run" to Streams application reset tool312 Add Overloaded StreamsBuilder Build Method to Accept java.util.Properties | 2.10.11.0.0 | ||||
| 111 | KIP-155 - Add range scan for windowed state stores309: Add toUpperCase support to sasl.kerberos.principal.to.local rule | 2.40.11.0.0 | ||||
| 112 | KIP-154 Add Kafka Connect configuration properties for creating internal topics308: Support dynamic update of max.connections.per.ip/max.connections.per.ip.overrides | 2.1.00.11.0.0 | ||||
| 113 | KIP- | 153: Include only client traffic in BytesOutPerSec metric307: Allow to define custom processor names with KStreams DSL | 2.3.0 (partial) | 0.11.0.0|||
| 114 | KIP- | 152 - Improve diagnostics for SASL authentication failures306: Configuration for Delaying Response to Failed Authentication | 2.1 | 1.0.0 | ||
| 115 | KIP-151 Expose Connector type in REST API305: Add Connect primitive number converters | 2.0.11.0.0 | ||||
| 116 | KIP- | 150 - Kafka-Streams Cogroup303: Add Dynamic Routing in Streams Sink | 2. | 1.0 (WIP)0.0 | ||
| 117 | KIP | -149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner-302 - Enable Kafka clients to use all DNS resolved IP addresses | 1.1.0 (partially implemented)2.1.0 | )|||
| 118 | KIP- | 146 - Classloading Isolation in Connect300: Add Windowed KTable API in StreamsBuilder | 3. | 0.11.0.0 (WIP) | ||
| 119 | KIP- | 145 - Expose Record Headers in Kafka298: Error Handling in Connect | 12. | 10.0 | ||
| 120 | KIP- | 144297: | Exponential backoff for broker reconnect attemptsExternalizing Secrets for Connect Configurations | 2. | 0.11.0.0 | |
| 121 | KIP- | 143: Controller Health Metrics295 Add Streams Configuration Allowing for Optional Topology Optimization | 2 | 0.11.0.0 | ||
| 122 | KIP | -140: Add administrative RPCs for adding, deleting, and listing ACLs-294 - Enable TLS hostname verification by default | 2. | 0.11.0.0 | ||
| 123 | 138: Change punctuate semantics | Accepted1.0.0 | ||||
| 124 | KIP- | 137: Enhance TopicCommand --describe to show topics marked for deletion292: Add transformValues() method to KTable | 2. | 0.11.0.0 | ||
| 125 | KIP- | 136: Add Listener name to SelectorMetrics tags290: Support for Prefixed ACLs | 2 | 0.11.0.0 | ||
| 126 | KIP- | 134: Delay initial consumer group rebalance289: Improve the default group id behavior in KafkaConsumer | 2.2.0 | 0.11.0.0|||
| 127 | KIP-133: Describe and Alter Configs Admin APIs285: Connect Rest Extension Plugin | 2.0.11.0.0 | ||||
| 128 | KIP-130: Expose states of active tasks to KafkaStreams public API284: Set default retention ms for Streams repartition topics to Long.MAX_VALUE | 21.0.0 | ||||
| 129 | KIP- | 129: Streams Exactly-Once Semantics283: Efficient Memory Usage for Down-Conversion | 2. | 0.11.0.0 | ||
| 130 | KIP- | 128282: Add | ByteArrayConverter for Kafka Connectthe listener name to the authentication context | 2. | 0.11.0.0 | |
| 131 | KIP- | 126 - Allow KafkaProducer to split and resend oversized batches.281: ConsumerPerformance: Increase Polling Loop Timeout and Make It Reachable by the End User | 2 | 0.11.0.0 | ||
| 132 | KIP-124 - Request rate quotas279: Fix log divergence between leader and follower after fast leader fail over | 20.11.0.0 | ||||
| 133 | KIP- | 123: Allow per stream/table timestamp extractor278 - Add version option to Kafka's commands | 2. | 0.11.0.0 | ||
| 134 | KIP | -122: Add Reset Consumer Group Offsets tooling-277 - Fine Grained ACL for CreateTopics API | 2 | 0.11.0.0 | ||
| 135 | KIP-121: Add KStream peek method276 - Add StreamsConfig prefix for different consumers | 2.0.11.0.0 | ||||
| 136 | KIP-120274: Cleanup Kafka Streams builder APISkipped Records Metrics | 21.0.0 | ||||
| 137 | KIP- | 119: Drop Support for Scala 2.10 in Kafka 0.11272: Add API version tag to broker's RequestsPerSec metric | 2 | 0.11.0.0 | ||
| 138 | KIP | -118: Drop Support for Java 7-270 - A Scala Wrapper Library for Kafka Streams | 2.0.0 | |||
| 139 | KIP-117: Add a public AdminClient API for Kafka admin operations268: Simplify Kafka Streams Rebalance Metadata Upgrade | 2.0.11.0.0 | ||||
| 140 | KIP-267: Add Processor Unit Test Support to Kafka Streams Test Utils | 2.115: Enforce offsets.topic.replication.factor upon __consumer_offsets auto topic creation0.11.0.0 | ||||
| 141 | KIP- | 114: KTable state stores and improved semantics266: Fix consumer indefinite blocking behavior | 2 | 0.11.0.0 | ||
| 142 | KIP-113: Support replicas movement between log directories265: Make Windowed Serde to public APIs | 2.01.1.0 | ||||
| 143 | KIP-112: Handle disk failure for JBOD261: Add Single Value Fetch in Window Stores | 21.0.0 | ||||
| 144 | KIP- | 109: Old Consumer Deprecation258: Allow to Store Record Timestamps in RocksDB | 2.3.0 (partially implemented) | 0.11.0.0|||
| 145 | KIP-108: Create Topic Policy-257 - Configurable Quota Management | 2.00.10.2.0 | ||||
| 146 | KIP- | 107: Add deleteRecordsBefore() API in AdminClient255: OAuth Authentication via SASL/OAUTHBEARER | 2 | 0.11.0.0 | ||
| 147 | KIP-251: Allow timestamp manipulation in Processor API | 2106 - Change Default unclean.leader.election.enabled from True to False0.11.0.0 | ||||
| 148 | KIP-105: Addition of Record Level for Sensors249: Add Delegation Token Operations to KafkaAdminClient | 2.0.0.10.2.0 | ||||
| 149 | KIP-104: Granular Sensors for Streams 247: Add public test utils for Kafka Streams | 1.10.10.2.0 | ||||
| 150 | KIP-103245: Separation of Internal and External trafficUse Properties instead of StreamsConfig in KafkaStreams constructor | 2.0.0.10.2.0 | ||||
| 151 | KIP- | 102 - Add close with timeout for consumers244: Add Record Header support to Kafka Streams Processor API | 2.0 | 0.10.2.0 | ||
| 152 | KIP-101 - Alter Replication Protocol to use Leader Epoch rather than High Watermark for Truncation243: Make ProducerConfig and ConsumerConfig constructors public | 1.10.11.0.0 | ||||
| 153 | KIP-100 - Relax Type constraints in Kafka Streams API239 Add queryableStoreName() to GlobalKTable | 1.10.10.2.0 | ||||
| 154 | KIP-99: Add Global Tables to Kafka Streams238: Expose Kafka cluster ID in Connect REST API | 1.10.10.2.0 | ||||
| 155 | KIP-98 - Exactly Once Delivery and Transactional Messaging237: More Controller Health Metrics | 2.0.11.0.0 | ||||
| 156 | KIP- | 97: Improved Kafka Client RPC Compatibility Policy235: Add DNS alias support for secured connection | 2.1. | 0.10.2.0 | ||
| 157 | KIP-96 - Add per partition metrics for in-sync and assigned replica count233: Simplify StreamsBuilder#addGlobalStore | 1.1.0.10.2.0 | ||||
| 158 | KIP- | 94 Session Windows231: Improve the Required ACL of ListGroups API | 2.1. | 0.10.2.0 | ||
| 159 | KIP-93: Improve invalid timestamp handling in Kafka Streams229: DeleteGroups API | 1.10.10.2.0 | ||||
| 160 | KIP-92 - Add per partition lag metrics to KafkaConsumer227 - Introduce Incremental FetchRequests to Increase Partition Scalability | 1.1.0.10.2.0 | ||||
| 161 | KIP-91 Provide Intuitive User Timeouts in The Producer226 - Dynamic Broker Configuration | 12.1.0 | ||||
| 162 | KIP-90 - Remove zkClient dependency from Streams225 - Use tags for consumer “records.lag” metrics | 1.10.10.2.0 | ||||
| 163 | KIP-89: Allow sink connectors to decouple flush and offset commit224: Add configuration parameter `retries` to Streams API | 1.1.0.10.2.0 | ||||
| 164 | KIP-88: OffsetFetch Protocol Update-223 - Add per-topic min lead and per-partition lead metrics to KafkaConsumer | 2.00.10.2.0 | ||||
| 165 | KIP-86: Configurable SASL callback handlers-222 - Add Consumer Group operations to Admin API | 2.0.0 | ||||
| 166 | KIP-85: Dynamic JAAS configuration for Kafka clients221: Enhance DSL with Connecting Topic Creation and Repartition Hint | 2.4.0 (WIP)0.10.2.0 | ||||
| 167 | KIP-84: Support SASL SCRAM mechanisms220: Add AdminClient into Kafka Streams' ClientSupplier | 1.10.10.2.0 | ||||
| 168 | KIP-82 - Add Record Headers219 - Improve quota communication | 20.11.0.0 | ||||
| 169 | KIP-81: Bound Fetch memory usage in the consumer218: Make KafkaFuture.Function java 8 lambda compatible | 1.1.02.0.0 (WIP) | ||||
| 170 | KIP-79 - ListOffsetRequest/ListOffsetResponse v1 and add timestamp search methods to the new consumer215: Add topic regex support for Connect sinks | 10.10.1.0 | ||||
| 171 | KIP-78: Cluster Id | 0.10.1.0 | ||||
| 172 | KIP-77: Improve Kafka Streams Join Semantics | 0.10.2.0 | ||||
| 214: Add zookeeper.max.in.flight.requests config to the broker | 1.1.0 | |||||
| 172 | KIP-213 Support non-key joining in KTable | 2.4.0 (WIP) | ||||
| 173 | KIP-212: Enforce set of legal characters for connector names | 1 | 173 | KIP-75 - Add per-connector Converters | 0.10.1.0 | |
| 174 | KIP-74: Add Fetch Response Size Limit in Bytes211: Revise Expiration Semantics of Consumer Group Offsets | 20.10.1.0 | ||||
| 175 | KIP-73: Replication Quotas210 - Provide for custom error handling when Kafka Streams fails to produce | 10.10.1.0 | ||||
| 176 | KIP-72: Allow putting a bound on memory consumed by Incoming request 208: Add SSL support to Kafka Connect REST interface | 1.01.0 | ||||
| 177 | KIP-71: Enable log compaction and deletion to co-exist207: Offsets returned by ListOffsetsResponse should be monotonically increasing even during a partition leader change | 2.20.10.1.0 | ||||
| 178 | KIP- | 70: Revise Partition Assignment Semantics on New Consumer's Subscription Change206: Add support for UUID serialization and deserialization | 2 | 0.10.1.0 | ||
| 179 | KIP-67: Queryable state for Kafka Streams205: Add all() and range() API to ReadOnlyWindowStore | 10.10.1.0 | ||||
| 180 | KIP-66: Single Message Transforms for Kafka Connect | 0.10.2.0 / 0.11.0.0 | 204 : Adding records deletion operation to the new Admin Client API | 1.1.0 | ||
| 181 | KIP-203: Add toLowerCase support to sasl.kerberos.principal.to.local rule | 1 | 181 | KIP-65: Expose timestamps to Connect | 0.10.1.0 | |
| 182 | KIP-63: Unify store and downstream caching in streams202: Move merge() from StreamsBuilder to KStream | 1.0.0.10.1.0 | ||||
| 183 | KIP-62: Allow consumer to send heartbeats from a background thread198: Remove ZK dependency from Streams Reset Tool | 1.0.0.10.1.0 | ||||
| 184 | KIP-60 - Make Java client classloading more flexible197 Connect REST API should include the connector type when describing a connector | 1.0.0.10.1.0 | ||||
| 185 | KIP-58 - Make Log Compaction Point Configurable196: Add metrics to Kafka Connect framework | 1.0.0.10.1.0 | ||||
| 186 | KIP-57 - Interoperable LZ4 Framing195: AdminClient.createPartitions | 10.10.0.0 | ||||
| 187 | KIP-56: Allow cross origin HTTP requests on all HTTP methods192 : Provide cleaner semantics when idempotence is enabled | 1.0.10.0.0 | ||||
| 188 | KIP-55: Secure Quotas for Authenticated Users191: KafkaConsumer.subscribe() overload that takes just Pattern | 1.00.10.1.0 | ||||
| 189 | KIP-54: Sticky Partition Assignment Strategy190: Handle client-ids consistently between clients and brokers | 1.0.11.0.0 | ||||
| 190 | KIP-52: Connector Control APIs189: Improve principal builder interface and add support for SASL | 10.10.0.0 | ||||
| 191 | KIP-51 - List Connectors REST API188 - Add new metrics to support health checks | 10.10.0.0 | ||||
| 192 | KIP-187 - Add cumulative count metric for all Kafka rate metrics | 1.050 - Move Authorizer to o.a.k.common package0.10.1.0 | ||||
| 193 | KIP-48 Delegation token support for Kafka186: Increase offsets retention default to 7 days | 2.01.1.0 | ||||
| 194 | KIP-45 - Standardize all client sequence interaction on j.u.Collection.183 - Change PreferredReplicaLeaderElectionCommand to use AdminClient | 2.2.00.10.0.0 | ||||
| 195 | KIP-43: Kafka SASL enhancements182: Reduce Streams DSL overloads and allow easier use of custom storage engines | 1.0.10.0.0 | ||||
| 196 | KIP-42: Add Producer and Consumer Interceptors180: Add a broker metric specifying the number of consumer group rebalances in progress | 1.1.00.10.0.0 | ||||
| 197 | KIP-41177: Consumer Max Recordsperf tool should count rebalance time | 1.0.10.0.0 | ||||
| 198 | KIP- | 40: ListGroups and DescribeGroup176: Remove deprecated new-consumer option for tools | 2 | 0.9.0.0 | ||
| 199 | KIP-38: ZooKeeper Authentication175: Additional '--describe' views for ConsumerGroupCommand | 1.10.9.0.0 | ||||
| 200 | KIP-36 - Rack aware replica assignment174 - Deprecate and remove internal converter configs in WorkerConfig | 2.0.10.0.0 | ||||
| 201 | KIP-35 - Retrieving protocol version173: Add prefix to StreamsConfig to enable setting default internal topic configs | 10.10.0.0 | ||||
| 202 | KIP-33 - Add a time based log index171 - Extend Consumer Group Reset Offset for Stream Application | 1.0.10.1.0 | ||||
| 203 | KIP-32 - Add timestamps to Kafka message168: Add GlobalTopicCount and GlobalPartitionCount metric per cluster | 1.0.10.0.0 | ||||
| 204 | KIP-31 - Move to relative offsets in compressed message sets167: Add interface for the state store restoration process | 1.0.10.0.0 | ||||
| 205 | 28 a processor client UnderMinIsrPartitionCount and per-partition UnderMinIsr metrics | 10.10.0.0 | ||||
| 206 | 26 - Add Kafka Connect framework for data import/export | 1 | 0.9.0.0 | |||
| 207 | KIP-25 - System test improvements162: Enable topic deletion by default | 1.0.9.0.0 | ||||
| 208 | KIP-22 - Expose a Partitioner interface in the new producer161: streams deserialization exception handlers | 10.9.0.0 | ||||
| 209 | KIP-21 - Dynamic Configuration160: Augment KStream.print(), KStream.writeAsText() to allow users pass in extra parameters in the printed string | 10.9.0.0 (WIP) | ||||
| 210 | KIP- | 20 Enable log preallocate to improve consume performance under windows and some old Linux file system157 - Add consumer config options to streams reset tool | 1. | 0.9.0.0 | ||
| 211 | KIP- | 19 - Add a request timeout to NetworkClient156 Add option "dry run" to Streams application reset tool | 0. | 911.0.0 | ||
| 212 | KIP- | 16 - Automated Replica Lag Tuning155 - Add range scan for windowed state stores | 0.11 | 0.9.0.0 | ||
| 213 | KIP-15 - Add a close method with a timeout in the producer154 Add Kafka Connect configuration properties for creating internal topics | 0.911.0.0 | ||||
| 214 | KIP-13 - Quota Design153: Include only client traffic in BytesOutPerSec metric | 0.911.0.0 | ||||
| 215 | KIP-12 - Kafka Sasl/Kerberos and SSL implementation152 - Improve diagnostics for SASL authentication failures | 10.9.0.0 | ||||
| 216 | KIP-11 - Kafka Authorizer design151 Expose Connector type in REST API | 0.911.0.0 | ||||
| 217 | KIP-8 - Add a flush method to the producer API150 - Kafka-Streams Cogroup | 2.5.0 (WIP)0.9.0.0 | ||||
| 218 | KIP-4 - Metadata Protocol Changes149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner | 1.1.0 (partially implemented) (WIP for 2.4.0)0.10.0.0 | ||||
| 219 | KIP-4 - Command line and centralized administrative operations146 - Classloading Isolation in Connect | 0. 9.0.0, 0.10.011.0 , 0.10.1.0 | ||||
| 220 | KIP-3 - Mirror Maker Enhancement145 - Expose Record Headers in Kafka Connect | 1.1.00.9.0.0 | ||||
| 221 | KIP-2 - Refactor brokers to allow listening on multiple ports and IPs144: Exponential backoff for broker reconnect attempts | 0.911.0.0 | ||||
| 222 | KIP-1 - Remove support of request.required.acks143: Controller Health Metrics | 0.911.0.0 |
...
...
KIPs under discussion
Please insert new rows in sorted order (ascending by KIP number). Please move the adopted/accepted KIPs to the "Adopted KIPs" table above.
Dormant/inactive KIPs
Please insert new rows in sorted order (ascending by KIP number).
Discarded KIPs
Please insert new rows in sorted order (ascending by KIP number).
...
Discarded KIPs
| KIP | Comment |
|---|---|
| KIP-5 - Broker Configuration Management | (Superseded by KIP-21) |
| KIP-7 - Security - IP Filtering | |
| KIP-9 - SSL Support | Adopted via KAFKA-1690 but not via this KIP process |
| KIP-18 - JBOD Support | Superseded by KIP-112 and KIP-113 |
| KIP-24 - Remove ISR information from TopicMetadataRequest and add broker level metadata request | |
| KIP-29 - Add an IsrPropagateIntervalMs configuration to KafkaConfig | No longer need after KAFKA-2722 |
KIP-34 Add Partitioner Change Listener to Partitioner Interface for Multiple Use Case | Per feedback it is better do this parition() method and avoid Thread Coordination etc. |
| KIP-80: Kafka Rest Server | |
| KIP-83 - Allow multiple SASL authenticated Java clients in a single JVM process | After KIP-85, no interface changes, reduced to KAFKA-4180 |
| KIP-116: Add State Store Checkpoint Interval Configuration | Currently not needed as checkpointing can be done on commit interval. |
| KIP-127: Pluggable JAAS LoginModule configuration for SSL | Similar functionality can be implemented using a custom PrincipalBuilder. |
| KIP-132: Augment KStream.print to allow extra parameters in the printed string | Duplicated by KIP-160 |
| KIP-184: Rename LogCleaner and related classes to LogCompactor | A lot of configuraiton change will cause more trouble and doesn't seems to be worth it |
| KIP-111: Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker. | (covered by KIP-189) |
| KIP-147: Add missing type parameters to StateStoreSupplier factories and KGroupedStream/Table methods | (covered by KIP-182) |
| KIP-170: Enhanced TopicCreatePolicy and introduction of TopicDeletePolicy | superseded by KIP-201: Rationalising Policy interfaces |
| KIP 172: Add regular-expression topic support for sink connector | superseded by KIP-215: Add topic regex support for Connect sinks |
| KIP-179: Change ReassignPartitionsCommand to use AdminClient | |
| KIP-246: Connect producers and consumers should inherit worker configs | Withdrawn because proposal could not be made backward compatible with existing behavior |
| KIP-263: Allow broker to skip sanity check of inactive segments on broker startup | Withdrawn because the solution we agreed on does not require interface change |
| KIP-286: producer.send() should not block on metadata update | Withdrawn because the benefit of not having to wait for metadata is probably not worth the complexity added in producer. |
| KIP-288: [DISCARDED] Consumer.poll() timeout semantic change and new waitForAssignment method | discarded in deference to KIP-266 |
| KIP-327: Add describe all topics API to AdminClient | withdrawn in-favor of filter support in Metadata API and KIP-142 |
| KIP-344: The auto-generated client id should be passed to MetricsReporter | Bug fix so KIP not needed |
KIP Discussion Recordings
...