...
- Click
. Take the next available KIP number and give your proposal a descriptive heading. e.g. "KIP 42: Allow Infinite Retention With Bounded Disk Usage".Create from template templateName 54329345 templateId 54329345 title KIP-NEXT: Insert Title Here buttonLabel Create KIP - Fill in the sections as described above
- Start a [DISCUSS] thread on the Apache mailing list. Please ensure that the subject of the thread is of the format [DISCUSS] KIP-{your KIP number} {your KIP heading} The and the body contains a link to your new KIP. The discussion should happen on the mailing list, not on the wiki, since the wiki comment system doesn't work well for larger discussions. In the process of the discussion you may update the proposal. You should let people know the changes you are making. When you feel you have a finalized proposal
- Once the proposal is finalized call a [VOTE] to have the proposal adopted. These proposals are more serious than code changes and more serious even than release votes. The criteria for acceptance is lazy majority. The vote should remain open for at least 72 hours.
- Please update the KIP wiki page, and the index below, to reflect the current stage of the KIP after a vote. This acts as the permanent record indicating the result of the KIP (e.g., Accepted or Rejected). Also report the result of the KIP vote to the voting thread on the mailing list so the conclusion is clear.
KIP round-up
Next KIP Number: 434530
Use this number as the identifier for your KIP and increment this value.
Adopted KIPs
Please insert new rows in sorted order (descending by KIP number).
KIP (please keep this sorted by KIP number) | Release | ||||||
---|---|---|---|---|---|---|---|
1 | KIP-428: Add in-memory window store | 2.3.0 | 500: Replace ZooKeeper with a Self-Managed Metadata Quorum | ||||
2 | 2KIP- | 425482: | Add some Log4J Kafka Appender Properties for Producing to Secured BrokersThe Kafka Protocol should Support Optional Tagged Fields | 2. | 34.0 (WIP) | ||
3 | KIP-420: Add Single Value Fetch in Session Stores504 - Add new Java Authorizer Interface | 2.24.0 (WIP) | |||||
4 | KIP-412: Extend Admin API to support dynamic application log levels503: Add metric for number of topics marked for deletion | 2.34.0 (WIP) | |||||
5 | KIP-414: Expose Embedded ClientIds in Kafka Streams497: Add inter-broker API to alter ISR | 2.24.0 (WIP) | |||||
6 | KIP-402: Improve fairness in SocketServer processors496: Administrative API to delete consumer offsets | 2.24.0 (WIP) | |||||
7 | KIP- | 394: Require member.id for initial join group request495: Dynamically Adjust Log Levels in Connect | 2. | 24.0 (WIP) | |||
8 | KIP-393: Time windowed serde to properly deserialize changelog input topic488: Clean up Sum,Count,Total Metrics | 2.24.0 | |||||
9 | KIP-386: Standardize on Min/Avg/Max metrics' default value484: Expose metrics for group and transaction metadata loading duration | 2.24.0 (WIP) | |||||
10 | KIP-382: MirrorMaker 2.0481: SerDe Improvements for Connect Decimal type in JSON | 2.34.0 (WIP) | |||||
11 | KIP- | 380: Detect outdated control requests and bounced brokers using broker generation480: Sticky Partitioner | 2. | 24.0 | |||
12 | KIP- | 377479: | TopicCommandAdd Materialized to | use AdminClientJoin | 2. | 24.0(WIP) | |
13 | KIP-376: Implement AutoClosable on appropriate classes that want to be used in a try-with-resource statement476: Add Java AdminClient Interface | 2.24.0 | |||||
14 | KIP- | 374: Add '--help' option to all available Kafka CLI commands475: New Metrics to Measure Number of Tasks on a Connector | 2. | 24.0 (WIP) | |||
15 | KIP- | 372: Naming Repartition Topics for Joins and Grouping474: To deprecate WindowStore#put(key, value) | 2. | 14.0 (WIP) | |||
16 | KIP-371: Add a configuration to build custom SSL principal name471: Expose RocksDB Metrics in Kafka Streams | 2. 24.0 (WIP) | |||||
17 | KIP | 368: Allow SASL Connections to Periodically Re-Authenticate-467: Augment ProduceResponse error messaging for specific culprit records | 2. | 24.0 (WIP) | |||
18 | KIP-367 Introduce close(Duration) to Producer and AdminClient instead of close(long, TimeUnit)465: Add Consolidated Connector Endpoint to Connect REST API | 2.23.0 | |||||
19 | KIP- | 366: Make FunctionConversions deprecated464: Defaults for AdminClient#createTopic | 2. | 14.0 | |||
20 | KIP-365: Materialized, Serialized, Joined, Consumed and Produced with implicit Serde462: Use local thread id for KStreams | 2.13.0 | |||||
21 | KIPKIP-361: Add Consumer Configuration to Disable Auto Topic Creation461: Improve Replica Fetcher behavior at handling partition failure | 2.3.0 (WIP) | |||||
22 | KIP- | 359: Verify leader epoch in produce requests460: Admin Leader Election RPC | 2. | 34.0 | (WIP)|||
23 | KIP- | 358: Migrate Streams API to Duration instead of long ms times458: Connector Client Config Override Policy | 2. | 13.0 | |||
24 | KIP- | 357: Add support to list ACLs per principal455: Create an Administrative API for Replica Reassignment | 2. | 14.0 (WIP) | |||
25 | KIP-356: Add withCachingDisabled() to StoreBuilder454: Expansion of the ConnectClusterState interface | 2.13.0 | |||||
26 | KIP-353: Improve Kafka Streams Timestamp Synchronization453: Add close() method to RocksDBConfigSetter | 2.13.0 | |||||
27 | KIP- | 351449: Add | --under-min-isr option to describe topics commandconnector contexts to log messages in Connect workers | 2.3.0 | |||
28 | KIP-346: Improve LogCleaner behavior on error | 2.1 | 29 | KIP-342 Add support for custom SASL extensions in OAuthBearer authentication446: Add changelog topic configuration to KTable suppress | 2.14.0 (WIP) | ||
29 | KIP-341: Update Sticky Assignor's User Data Protocol445: In-memory Session Store | 2.3.0 (WIP) | |||||
3130 | 340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file | 2.1.0 | 2.3.0 | ||||
31 | KIP-442: Return to default max poll interval in Streams | 2.3.0 | |||||
32 | KIP-440: Extend Connect Converter to support headers | 2.4 | 32 | KIP-339: Create a new IncrementalAlterConfigs API | 2.3.0 (WIP) | ||
33 | KIP-338 Support to exclude the internal topics in kafka-topics.sh command436: Add a metric indicating start time | 2.13.0 | |||||
34 | KIP-336: Consolidate ExtendedSerializer/Serializer and ExtendedDeserializer/Deserializer434: Add Replica Fetcher and Log Cleaner Count Metrics | 2.14.0 (WIP) | |||||
35 | KIP | -332: Update AclCommand to use AdminClient API-430 - Return Authorized Operations in Describe Responses | 2. | 13.0 | |||
36 | KIP-429: Kafka Consumer Incremental Rebalance Protocol | 2.4.0 (331 Add default implementation to close() and configure() for Serializer, Deserializer and Serde2.3.0 (WIP) | |||||
37 | KIP- | 330428: Add | retentionPeriodin | SessionBytesStoreSupplier-memory window store | 2. | 13.0 | |
38 | KIP- | 328: Ability to suppress updates for KTables427: Add AtMinIsr topic partition category (new metric & TopicCommand option) | 2.1.0 (partially implemented) /2.3.0 | (WIP)||||
39 | KIP-324425: Add method to get metrics() in AdminClientsome Log4J Kafka Appender Properties for Producing to Secured Brokers | 2.13.0 | |||||
40 | KIP- | 322: Return new error code for DeleteTopics API when topic deletion disabled.421: Support resolving externalized secrets in AbstractConfig | 2. | 13.0 | |||
41 | KIP- | 321: Update TopologyDescription to better represent Source and Sink Nodes420: Add Single Value Fetch in Session Stores | 2. | 12.0 | |||
42 | KIP- | 320417: Allow | fetchers to detect and handle log truncationJmxTool to connect to a secured RMI port | 2. | 1.0 (partially implemented) / 2.23.0 | ||
43 | KIP- | 319: Replace segments with segmentInterval in WindowBytesStoreSupplier415: Incremental Cooperative Rebalancing in Kafka Connect | 2. | 13.0 | |||
44 | KIP-313: Add KStream.flatTransform and KStream.flatTransformValues 414: Expose Embedded ClientIds in Kafka Streams | 2.2.0 (partially implemented)/ 2.3 | |||||
45 | KIP-412: Extend Admin API to support dynamic application log levels | 2.4.0 (WIP) | 45|||||
46 | 312 Add Overloaded StreamsBuilder Build Method to Accept java.util.Properties | 2. | 13.0 | 46||||
47 | KIP- | 308: Support dynamic update of max.connections.per.ip/max.connections.per.ip.overrides402: Improve fairness in SocketServer processors | 2. | 1.047 | KIP-306: Configuration for Delaying Response to Failed Authentication | 2.12.0 (partially implemented) / 2.3.0 | |
48 | KIP- | 305: Add Connect primitive number converters401: TransformerSupplier/ProcessorSupplier StateStore connecting | 2. | 04.0 (WIP) | |||
49 | KIP-303396: Add Dynamic Routing in Streams SinkCommit/List Offsets Operations to AdminClient | 2.04.0 (WIP) | |||||
50 | KIP-302 - Enable Kafka clients to use all DNS resolved IP addresses394: Require member.id for initial join group request | 2.12.0 | |||||
51 | KIP-298: Error Handling in Connect393: Time windowed serde to properly deserialize changelog input topic | 2.02.0 | |||||
52 | KIP- | 297: Externalizing Secrets for Connect Configurations389: Introduce a configurable consumer group size limit | 2. | 02.0 | |||
53 | KIP-295 Add Streams Configuration Allowing for Optional Topology Optimization386: Standardize on Min/Avg/Max metrics' default value | 2.02.0 | |||||
54 | KIP-294 - Enable TLS hostname verification by default382: MirrorMaker 2.0 | 2.04.0 | |||||
55 | 292: Add transformValues() method to KTable | 2. | 04.0 (WIP) | ||||
56 | KIP-290: Support for Prefixed ACLs380: Detect outdated control requests and bounced brokers using broker generation | 2.02.0 | |||||
57 | KIP-289: Improve the default group id behavior in KafkaConsumer377: TopicCommand to use AdminClient | 2.2.0 | |||||
58 | KIP- | 285: Connect Rest Extension Plugin376: Implement AutoClosable on appropriate classes that want to be used in a try-with-resource statement | 2. | 02.0 | |||
59 | KIP-284: Set default retention ms for Streams repartition topics to Long.MAX_VALUE374: Add '--help' option to all available Kafka CLI commands | 2.02.0 | |||||
60 | KIP-283: Efficient Memory Usage for Down-Conversion372: Naming Repartition Topics for Joins and Grouping | 2.01.0 | |||||
61 | KIP- | 282371: Add | the listener name to the authentication contexta configuration to build custom SSL principal name | 2. | 02.0 | ||
62 | KIP- | 281: ConsumerPerformance: Increase Polling Loop Timeout and Make It Reachable by the End User369: Alternative Partitioner to Support "Always Round-Robin" Selection | 2. | 04.0 | |||
63 | KIP -279: Fix log divergence between leader and follower after fast leader fail over368: Allow SASL Connections to Periodically Re-Authenticate | 2.02.0 | |||||
64 | KIP-278 - Add version option to Kafka's commands367 Introduce close(Duration) to Producer and AdminClient instead of close(long, TimeUnit) | 2.02.0 | |||||
65 | KIP- | 277 - Fine Grained ACL for CreateTopics API366: Make FunctionConversions deprecated | 2. | 01.0 | |||
66 | KIP- | 276 - Add StreamsConfig prefix for different consumers365: Materialized, Serialized, Joined, Consumed and Produced with implicit Serde | 2. | 01.0 | |||
67 | KIPKIP- | 274: Kafka Streams Skipped Records Metrics361: Add Consumer Configuration to Disable Auto Topic Creation | 2. | 03.0 | |||
68 | KIP-272: Add API version tag to broker's RequestsPerSec metric360: Improve handling of unknown producer | 2.04.0 (WIP) | |||||
69 | KIP-270 - A Scala Wrapper Library for Kafka Streams359: Verify leader epoch in produce requests | 2.04.0 (WIP) | |||||
70 | KIP- | 268358: | Simplify Kafka Streams Rebalance Metadata UpgradeMigrate Streams API to Duration instead of long ms times | 2. | 01.0 | ||
71 | KIP-267357: Add Processor Unit Test Support to Kafka Streams Test Utilssupport to list ACLs per principal | 2.01.0 | |||||
72 | KIP-266: Fix consumer indefinite blocking behavior356: Add withCachingDisabled() to StoreBuilder | 2.01.0 | |||||
73 | KIP- | 265: Make Windowed Serde to public APIs354: Add a Maximum Log Compaction Lag | 2. | 03.0 | |||
74 | KIP- | 261: Add Single Value Fetch in Window Stores353: Improve Kafka Streams Timestamp Synchronization | 2. | 01.0 | |||
75 | KIP-258: Allow to Store Record Timestamps in RocksDB352: Distinguish URPs caused by reassignment | 2.34.0 (WIP) | |||||
76 | KIP- | 257 - Configurable Quota Management351: Add --under-min-isr option to describe topics command | 2. | 03.0 | |||
77 | KIP-255: OAuth Authentication via SASL/OAUTHBEARER346: Improve LogCleaner behavior on error | 2.0.01 | |||||
78 | KIP- | 251: Allow timestamp manipulation in Processor API345: Introduce static membership protocol to reduce consumer rebalances | 2.4.0 (WIP), partially available in 2.3.0 | ||||
79 | KIP- | 249: Add Delegation Token Operations to KafkaAdminClient342 Add support for custom SASL extensions in OAuthBearer authentication | 2. | 01.0 | |||
80 | KIP- | 247: Add public test utils for Kafka Streams341: Update Sticky Assignor's User Data Protocol | 2.3 | 1.1.0 | |||
81 | KIP- | 245: Use Properties instead of StreamsConfig in KafkaStreams constructor340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file | 2. | 01.0 | |||
82 | KIP-244: Add Record Header support to Kafka Streams Processor 339: Create a new IncrementalAlterConfigs API | 2.03.00 | |||||
83 | KIP- | 243: Make ProducerConfig and ConsumerConfig constructors public338 Support to exclude the internal topics in kafka-topics.sh command | 2 | 1.1.0 | |||
84 | KIP- | 239 Add queryableStoreName() to GlobalKTable336: Consolidate ExtendedSerializer/Serializer and ExtendedDeserializer/Deserializer | 2 | 1.1.0 | |||
85 | KIP-238: Expose Kafka cluster ID in Connect REST API332: Update AclCommand to use AdminClient API | 21.1.0 | |||||
86 | KIP- | 237: More Controller Health Metrics331 Add default implementation to close() and configure() for Serializer, Deserializer and Serde | 2. | 03.0 | |||
87 | KIP-235330: Add DNS alias support for secured connectionretentionPeriod in SessionBytesStoreSupplier | 2.1.0 | |||||
88 | KIP- | 233: Simplify StreamsBuilder#addGlobalStore328: Ability to suppress updates for KTables | 2.1.0 (partially implemented) / 2.3.0 (WIP) | 1.1.0||||
89 | KIP-231: Improve the Required ACL of ListGroups API324: Add method to get metrics() in AdminClient | 2.1.0 | |||||
90 | KIP-229: DeleteGroups API322: Return new error code for DeleteTopics API when topic deletion disabled. | 21.1.0 | |||||
91 | KIP- | 227 - Introduce Incremental FetchRequests to Increase Partition Scalability321: Update TopologyDescription to better represent Source and Sink Nodes | 2 | 1.1.0 | |||
92 | KIP- | 226 - Dynamic Broker Configuration320: Allow fetchers to detect and handle log truncation | 2.1.0 (partially implemented) / 2.2 | 1.1.0 | |||
93 | KIP- | 225 - Use tags for consumer “records.lag” metrics319: Replace segments with segmentInterval in WindowBytesStoreSupplier | 2 | 1.1.0 | |||
94 | KIP- | 224: Add configuration parameter `retries` to Streams API313: Add KStream.flatTransform and KStream.flatTransformValues | 2.2.0 (partially implemented) / 2.3 | 1.1.0 | |||
95 | KIP- | 223 - Add per-topic min lead and per-partition lead metrics to KafkaConsumer312 Add Overloaded StreamsBuilder Build Method to Accept java.util.Properties | 2. | 01.0 | |||
96 | KIP- | 222 - Add Consumer Group operations to Admin API309: Add toUpperCase support to sasl.kerberos.principal.to.local rule | 2. | 04.0 | |||
97 | KIP-220: Add AdminClient into Kafka Streams' ClientSupplier308: Support dynamic update of max.connections.per.ip/max.connections.per.ip.overrides | 21.1.0 | |||||
98 | KIP- | 219 - Improve quota communication307: Allow to define custom processor names with KStreams DSL | 2. | 03.0 (partial) | |||
99 | KIP- | 218: Make KafkaFuture.Function java 8 lambda compatible306: Configuration for Delaying Response to Failed Authentication | 2 | 1.1.0 | |||
100 | KIP-215305: Add topic regex support for Connect sinksConnect primitive number converters | 2.01.1.0 | |||||
101 | KIP- | 214303: Add | zookeeper.max.in.flight.requests config to the brokerDynamic Routing in Streams Sink | 2.0 | 1.1.0 | ||
102 | KIP | -212: Enforce set of legal characters for connector names-302 - Enable Kafka clients to use all DNS resolved IP addresses | 2 | 1.1.0 | |||
103 | KIP- | 211: Revise Expiration Semantics of Consumer Group Offsets300: Add Windowed KTable API in StreamsBuilder | 2. | 14.0 | |||
104 | KIP- | 210 - Provide for custom error handling when Kafka Streams fails to produce298: Error Handling in Connect | 2.0 | 1.1.0 | |||
105 | KIP- | 208: Add SSL support to Kafka Connect REST interface297: Externalizing Secrets for Connect Configurations | 2.0 | 1.1.0 | |||
106 | KIP- | 207: Offsets returned by ListOffsetsResponse should be monotonically increasing even during a partition leader change295 Add Streams Configuration Allowing for Optional Topology Optimization | 2. | 20.0 | |||
107 | KIP-206: Add support for UUID serialization and deserialization-294 - Enable TLS hostname verification by default | 2.10.0 | |||||
108 | 205 | Accepted | |||||
109 | KIP-292: Add | alltransformValues() | and range() API to ReadOnlyWindowStore1.1method to KTable | 2.0.0 | |||
109110 | KIP- | 204 : Adding records deletion operation to the new Admin Client API290: Support for Prefixed ACLs | 2.0 | 1.1.0 | |||
110111 | KIP-289: Improve the default group id behavior in KafkaConsumer | 2.2.0 | |||||
112 | KIP-285: Connect Rest Extension Plugin | 2 | 203: Add toLowerCase support to sasl.kerberos.principal.to.local rule1.1.0 | 111 | KIP-202: Move merge() from StreamsBuilder to KStream | 1.0.0 | |
112113 | KIP- | 198: Remove ZK dependency from Streams Reset Tool284: Set default retention ms for Streams repartition topics to Long.MAX_VALUE | 2 | 1.0.0 | |||
113114 | KIP- | 197 Connect REST API should include the connector type when describing a connector283: Efficient Memory Usage for Down-Conversion | 2 | 1.0.0 | |||
114115 | KIP- | 196282: Add | metrics to Kafka Connect frameworkthe listener name to the authentication context | 2 | 1.0.0 | ||
115116 | KIP- | 195: AdminClient.createPartitions281: ConsumerPerformance: Increase Polling Loop Timeout and Make It Reachable by the End User | 2 | 1.0.0 | 116 | KIP-192 : Provide cleaner semantics when idempotence is enabled | 1.0.0 |
117 | KIP-191: KafkaConsumer.subscribe() overload that takes just Pattern279: Fix log divergence between leader and follower after fast leader fail over | 21.0.0 | |||||
118 | KIP- | 190: Handle client-ids consistently between clients and brokers278 - Add version option to Kafka's commands | 2 | 1.0.0 | |||
119 | KIP | -189: Improve principal builder interface and add support for SASL-277 - Fine Grained ACL for CreateTopics API | 2 | 1.0.0 | |||
120 | KIP-188 276 - Add new metrics to support health checksStreamsConfig prefix for different consumers | 21.0.0 | |||||
121 | KIP-187 - Add cumulative count metric for all Kafka rate metrics274: Kafka Streams Skipped Records Metrics | 21.0.0 | |||||
122 | KIP- | 186: Increase offsets retention default to 7 days272: Add API version tag to broker's RequestsPerSec metric | 2.0.0 | ||||
123 | KIP- | 183 - Change PreferredReplicaLeaderElectionCommand to use AdminClient270 - A Scala Wrapper Library for Kafka Streams | 2. | 20.0 | |||
124 | KIP-182268: Reduce Streams DSL overloads and allow easier use of custom storage enginesSimplify Kafka Streams Rebalance Metadata Upgrade | 21.0.0 | |||||
125 | KIP-180: Add a broker metric specifying the number of consumer group rebalances in progress267: Add Processor Unit Test Support to Kafka Streams Test Utils | 2.01.1.0 | |||||
126 | KIP- | 177: Consumer perf tool should count rebalance time266: Fix consumer indefinite blocking behavior | 2 | 1.0.0 | |||
127 | KIP- | 176: Remove deprecated new-consumer option for tools265: Make Windowed Serde to public APIs | 2.0.0 | ||||
128 | KIP-175: Additional '--describe' views for ConsumerGroupCommand261: Add Single Value Fetch in Window Stores | 2.01.1.0 | |||||
129 | KIP- | 174 - Deprecate and remove internal converter configs in WorkerConfig258: Allow to Store Record Timestamps in RocksDB | 2. | 03.0 (partially implemented) | |||
130 | KIP-173: Add prefix to StreamsConfig to enable setting default internal topic configs257 - Configurable Quota Management | 21.0.0 | |||||
131 | KIP- | 171 - Extend Consumer Group Reset Offset for Stream Application255: OAuth Authentication via SASL/OAUTHBEARER | 2.0 | 1.1.0 | |||
132 | KIP-168: Add GlobalTopicCount and GlobalPartitionCount metric per cluster251: Allow timestamp manipulation in Processor API | 21.0.0 | |||||
133 | KIP-167249: Add interface for the state store restoration processDelegation Token Operations to KafkaAdminClient | 21.0.0 | |||||
134 | KIP-164 - Add UnderMinIsrPartitionCount and per-partition UnderMinIsr metrics247: Add public test utils for Kafka Streams | 1.01.0 | |||||
135 | KIP- | 163: Lower the Minimum Required ACL Permission of OffsetFetch245: Use Properties instead of StreamsConfig in KafkaStreams constructor | 2 | 1.0.0 | |||
136 | KIP- | 162: Enable topic deletion by default244: Add Record Header support to Kafka Streams Processor API | 2 | 1.0.0 | |||
137 | KIP-161: streams deserialization exception handlers243: Make ProducerConfig and ConsumerConfig constructors public | 1.01.0 | |||||
138 | KIP-160: Augment KStream.print(), KStream.writeAsText239 Add queryableStoreName() to allow users pass in extra parameters in the printed stringGlobalKTable | 1.01.0 | |||||
139 | KIP- | 157 - Add consumer config options to streams reset tool238: Expose Kafka cluster ID in Connect REST API | 1. | 01.0 | |||
140 | KIP- | 156 Add option "dry run" to Streams application reset tool237: More Controller Health Metrics | 2. | 0.11.0.0 | |||
141 | KIP-155 - Add range scan for windowed state stores235: Add DNS alias support for secured connection | 2.10.11.0.0 | |||||
142 | KIP-154 Add Kafka Connect configuration properties for creating internal topics233: Simplify StreamsBuilder#addGlobalStore | 1.10.11.0.0 | |||||
143 | KIP- | 153: Include only client traffic in BytesOutPerSec metric231: Improve the Required ACL of ListGroups API | 2.1.0 | 0.11.0.0||||
144 | KIP-152 - Improve diagnostics for SASL authentication failures229: DeleteGroups API | 1.01.0 | |||||
145 | KIP-151 Expose Connector type in REST API227 - Introduce Incremental FetchRequests to Increase Partition Scalability | 1.10.11.0.0 | |||||
146 | KIP-150 - Kafka-Streams Cogroup226 - Dynamic Broker Configuration | 1.1.02.3.0 (WIP) | |||||
147 | KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner225 - Use tags for consumer “records.lag” metrics | 1.1.0 (partially implemented)(WIP for 2.3.0) | |||||
148 | KIP-146 - Classloading Isolation in Connect224: Add configuration parameter `retries` to Streams API | 1.10.11.0.0 | |||||
149 | KIP-145 - Expose Record Headers in Kafka Connect223 - Add per-topic min lead and per-partition lead metrics to KafkaConsumer | 2.01.1.0 | |||||
150 | KIP-144: Exponential backoff for broker reconnect attempts222 - Add Consumer Group operations to Admin API | 20.11.0.0 | |||||
151 | KIP-143: Controller Health Metrics221: Enhance DSL with Connecting Topic Creation and Repartition Hint | 2.4.0 (WIP)0.11.0.0 | |||||
152 | KIP-140: Add administrative RPCs for adding, deleting, and listing ACLs220: Add AdminClient into Kafka Streams' ClientSupplier | 1.10.11.0.0 | |||||
153 | KIP-138: Change punctuate semantics219 - Improve quota communication | 21.0.0 | |||||
154 | KIP-137: Enhance TopicCommand --describe to show topics marked for deletion218: Make KafkaFuture.Function java 8 lambda compatible | 1.10.11.0.0 | |||||
155 | KIP-136215: Add Listener name to SelectorMetrics tagstopic regex support for Connect sinks | 1.10.11.0.0 | |||||
156 | KIP-134: Delay initial consumer group rebalance214: Add zookeeper.max.in.flight.requests config to the broker | 1.10.11.0.0 | |||||
157 | KIP- | 133: Describe and Alter Configs Admin APIs213 Support non-key joining in KTable | 2.4.0 (WIP) | 0.11.0.0||||
158 | KIP- | 130212: | Expose states of active tasks to KafkaStreams public APIEnforce set of legal characters for connector names | 1. | 01.0 | ||
159 | KIP-129: Streams Exactly-Once Semantics211: Revise Expiration Semantics of Consumer Group Offsets | 2.1.00.11.0.0 | |||||
160 | KIP-128: Add ByteArrayConverter for Kafka Connect210 - Provide for custom error handling when Kafka Streams fails to produce | 1.10.11.0.0 | |||||
161 | KIP-126 - Allow KafkaProducer to split and resend oversized batches.208: Add SSL support to Kafka Connect REST interface | 1.10.11.0.0 | |||||
162 | KIP-124 - Request rate quotas207: Offsets returned by ListOffsetsResponse should be monotonically increasing even during a partition leader change | 2.2.00.11.0.0 | |||||
163 | KIP- | 123: Allow per stream/table timestamp extractor206: Add support for UUID serialization and deserialization | 2.1 | 0.11.0.0 | |||
164 | KIP-122: Add Reset Consumer Group Offsets tooling205: Add all() and range() API to ReadOnlyWindowStore | 1.10.11.0.0 | |||||
165 | KIP-121: Add KStream peek method204 : Adding records deletion operation to the new Admin Client API | 1.10.11.0.0 | |||||
166 | KIP-120: Cleanup Kafka Streams builder API203: Add toLowerCase support to sasl.kerberos.principal.to.local rule | 1.01.0 | |||||
167 | KIP-202: Move merge() from StreamsBuilder to KStream | 1119: Drop Support for Scala 2.10 in Kafka 0.110.11.0.0 | |||||
168 | KIP-118: Drop Support for Java 7198: Remove ZK dependency from Streams Reset Tool | 12.0.0 | |||||
169 | KIP-117: Add a public AdminClient API for Kafka admin operations197 Connect REST API should include the connector type when describing a connector | 10.11.0.0 | |||||
170 | KIP-115: Enforce offsets.topic.replication.factor upon __consumer_offsets auto 196: Add metrics to Kafka Connect framework | 1 topic creation0.11.0.0 | |||||
171 | KIP-114: KTable state stores and improved semantics195: AdminClient.createPartitions | 10.11.0.0 | |||||
172 | KIP-113: Support replicas movement between log directories192 : Provide cleaner semantics when idempotence is enabled | 1.10.0 | |||||
173 | KIP-112: Handle disk failure for JBOD191: KafkaConsumer.subscribe() overload that takes just Pattern | 1.0.0 | |||||
174 | KIP- | 110: Add Codec for ZStandard Compression190: Handle client-ids consistently between clients and brokers | 1.0 | 2.1.0 | |||
175 | KIP-109: Old Consumer Deprecation189: Improve principal builder interface and add support for SASL | 10.11.0.0 | |||||
176 | KIP-108: Create Topic Policy-188 - Add new metrics to support health checks | 1.00.10.2.0 | |||||
177 | KIP-187 - 107: Add deleteRecordsBefore() API in AdminClientAdd cumulative count metric for all Kafka rate metrics | 1.0.11.0.0 | |||||
178 | KIP-106 - Change Default unclean.leader.election.enabled from True 186: Increase offsets retention default to 7 days | 2. to False0.11.0.0 | |||||
179 | KIP-105: Addition of Record Level for Sensors183 - Change PreferredReplicaLeaderElectionCommand to use AdminClient | 20.10.2.0 | |||||
180 | KIP-104182: Granular Sensors for Streams Reduce Streams DSL overloads and allow easier use of custom storage engines | 1.0.0.10.2.0 | |||||
181 | KIP-103: Separation of Internal and External traffic180: Add a broker metric specifying the number of consumer group rebalances in progress | 1.10.10.2.0 | |||||
182 | KIP-102 - Add close with timeout for consumers177: Consumer perf tool should count rebalance time | 1.0.0.10.2.0 | |||||
183 | KIP- | 101 - Alter Replication Protocol to use Leader Epoch rather than High Watermark for Truncation176: Remove deprecated new-consumer option for tools | 2. | 0.11.0.0 | |||
184 | KIP-100 - Relax Type constraints in Kafka Streams API175: Additional '--describe' views for ConsumerGroupCommand | 1.10.10.2.0 | |||||
185 | KIP-99: Add Global Tables to Kafka Streams-174 - Deprecate and remove internal converter configs in WorkerConfig | 2.00.10.2.0 | |||||
186 | KIP-98 - Exactly Once Delivery and Transactional Messaging173: Add prefix to StreamsConfig to enable setting default internal topic configs | 10.11.0.0 | |||||
187 | KIP-97: Improved Kafka Client RPC Compatibility Policy-171 - Extend Consumer Group Reset Offset for Stream Application | 1.10.10.2.0 | |||||
188 | KIP-96 - Add per partition metrics for in-sync and assigned replica count168: Add GlobalTopicCount and GlobalPartitionCount metric per cluster | 1.0.0.10.2.0 | |||||
189 | KIP-94 Session Windows167: Add interface for the state store restoration process | 1.00.10.2.0 | |||||
190 | -93: Improve invalid timestamp handling in Kafka Streams -164 - Add UnderMinIsrPartitionCount and per-partition UnderMinIsr metrics | 1.00.10.2.0 | |||||
191 | 92 - Add per partition lag metrics to KafkaConsumer | 1.0 | 0.10.2.0 | ||||
192 | KIP-91 Provide Intuitive User Timeouts in The Producer162: Enable topic deletion by default | 1.02.1.0 | |||||
193 | KIP-90 - Remove zkClient dependency from Streams161: streams deserialization exception handlers | 1.0.0.10.2.0 | |||||
194 | KIP-89: Allow sink connectors to decouple flush and offset commit160: Augment KStream.print(), KStream.writeAsText() to allow users pass in extra parameters in the printed string | 1.00.10.2.0 | |||||
195 | KIP | -88: OffsetFetch Protocol Update-157 - Add consumer config options to streams reset tool | 1.0. | 0.10.2.0 | |||
196 | KIP- | 86: Configurable SASL callback handlers156 Add option "dry run" to Streams application reset tool | 0.11 | 2.0.0 | |||
197 | KIP- | 85: Dynamic JAAS configuration for Kafka clients155 - Add range scan for windowed state stores | 0. | 1011. | 20.0 | ||
198 | KIP-84: Support SASL SCRAM mechanisms154 Add Kafka Connect configuration properties for creating internal topics | 0.1011.20.0 | |||||
199 | KIP-82 - Add Record Headers153: Include only client traffic in BytesOutPerSec metric | 0.11.0.0 | |||||
200 | KIP-81: Bound Fetch memory usage in the consumer152 - Improve diagnostics for SASL authentication failures | 1.0.02.2.0 (WIP) | |||||
201 | KIP-79 - ListOffsetRequest/ListOffsetResponse v1 and add timestamp search methods to the new consumer151 Expose Connector type in REST API | 0.1011.10.0 | |||||
202 | KIP-78: Cluster Id-150 - Kafka-Streams Cogroup | 2.4.0 (WIP)0.10.1.0 | |||||
203 | KIP-77: Improve Kafka Streams Join Semantics149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner | 1.1.0 (partially implemented) (WIP for 2.4.0)0.10.2.0 | |||||
204 | KIP-75 - Add per-connector Converters146 - Classloading Isolation in Connect | 0. 1011. 10.0 | |||||
205 | KIP-74: Add Fetch Response Size Limit in Bytes-145 - Expose Record Headers in Kafka Connect | 1.0.10.1.0 | |||||
206 | KIP-73: Replication Quotas144: Exponential backoff for broker reconnect attempts | 0.1011.10.0 | |||||
207 | KIP-72: Allow putting a bound on memory consumed by Incoming request 143: Controller Health Metrics | 0.111.0.0 | |||||
208 | KIP-71: Enable log compaction and deletion to co-exist140: Add administrative RPCs for adding, deleting, and listing ACLs | 0.1011.10.0 | |||||
209 | KIP-70: Revise Partition Assignment Semantics on New Consumer's Subscription Change138: Change punctuate semantics | 1.0.0.10.1.0 | |||||
210 | KIP-67: Queryable state for Kafka Streams137: Enhance TopicCommand --describe to show topics marked for deletion | 0.1011.10.0 | |||||
211 | KIP-66: Single Message Transforms for Kafka Connect136: Add Listener name to SelectorMetrics tags | 0.1011.20.0 / 0.11.0.0 | |||||
212 | KIP-65: Expose timestamps to Connect134: Delay initial consumer group rebalance | 0.1011.10.0 | |||||
213 | KIP-63133: Unify store and downstream caching in streamsDescribe and Alter Configs Admin APIs | 0.1011.10.0 | |||||
214 | KIP- | 62: Allow consumer to send heartbeats from a background thread130: Expose states of active tasks to KafkaStreams public API | 1.0 | 0.10.1.0 | |||
215 | KIP-60 - Make Java client classloading more flexible129: Streams Exactly-Once Semantics | 0.1011.10.0 | |||||
216 | KIP-58 - Make Log Compaction Point Configurable128: Add ByteArrayConverter for Kafka Connect | 0.1011.10.0 | |||||
217 | KIP-57 - Interoperable LZ4 Framing126 - Allow KafkaProducer to split and resend oversized batches. | 0.1011.0.0 | |||||
218 | KIP-56: Allow cross origin HTTP requests on all HTTP methods-124 - Request rate quotas | 0.1011.0.0 | |||||
219 | KIP-55: Secure Quotas for Authenticated Users123: Allow per stream/table timestamp extractor | 0.1011.10.0 | |||||
220 | KIP-54: Sticky Partition Assignment Strategy122: Add Reset Consumer Group Offsets tooling | 0.11.0.0 | |||||
221 | KIP-52: Connector Control APIs121: Add KStream peek method | 0. 1011.0.0 | |||||
222 | KIP-51 - List Connectors REST 120: Cleanup Kafka Streams builder API | 01.10.0.0 | |||||
223 | KIP-50 - Move Authorizer to o.a.k.common package119: Drop Support for Scala 2.10 in Kafka 0.11 | 0.1011.10.0 | |||||
224 | KIP-48 Delegation token support for Kafka118: Drop Support for Java 7 | 2.01.1.0 | |||||
225 | KIP-45 - Standardize all client sequence interaction on j.u.Collection.117: Add a public AdminClient API for Kafka admin operations | 0.1011.0.0 | |||||
226 | KIP-43: Kafka SASL enhancements115: Enforce offsets.topic.replication.factor upon __consumer_offsets auto topic creation | 0.1011.0.0 | |||||
227 | KIP-42: Add Producer and Consumer Interceptors114: KTable state stores and improved semantics | 0.1011.0.0 | |||||
228 | KIP-41: Consumer Max Records113: Support replicas movement between log directories | 1.10.10.0.0 | |||||
229 | KIP-40: ListGroups and DescribeGroup112: Handle disk failure for JBOD | 10.9.0.0 | |||||
230 | KIP- | 38: ZooKeeper Authentication110: Add Codec for ZStandard Compression | 2.1 | 0.9.0.0 | |||
231 | KIP-36 - Rack aware replica assignment109: Old Consumer Deprecation | 0.1011.0.0 | |||||
232 | KIP-35 - Retrieving protocol version108: Create Topic Policy | 0.10.02.0 | |||||
233 | KIP-33 - Add a time based log index107: Add deleteRecordsBefore() API in AdminClient | 0.1011.10.0 | |||||
234 | 32 - Add timestamps to Kafka message 106 - Change Default unclean.leader.election.enabled from True to False | 0.1011.0.0 | |||||
235 | KIP-31 - Move to relative offsets in compressed message sets105: Addition of Record Level for Sensors | 0.10.02.0 | |||||
236 | KIP-28 - Add a processor client104: Granular Sensors for Streams | 0.10.02.0 | |||||
237 | KIP-26 - Add Kafka Connect framework for data import/export103: Separation of Internal and External traffic | 0.910.02.0 | |||||
238 | KIP-25 - System test improvements102 - Add close with timeout for consumers | 0.910.02.0 | |||||
239 | KIP-22 - Expose a Partitioner interface in the new producer101 - Alter Replication Protocol to use Leader Epoch rather than High Watermark for Truncation | 0.911.0.0 | |||||
240 | KIP-21 - Dynamic Configuration100 - Relax Type constraints in Kafka Streams API | 0.910.02.0 (WIP) | |||||
241 | KIP-20 Enable log preallocate to improve consume performance under windows and some old Linux file system99: Add Global Tables to Kafka Streams | 0.910.02.0 | |||||
242 | KIP-19 - Add a request timeout to NetworkClient98 - Exactly Once Delivery and Transactional Messaging | 0.911.0.0 | |||||
243 | KIP-16 - Automated Replica Lag Tuning97: Improved Kafka Client RPC Compatibility Policy | 0.910.02.0 | |||||
244 | KIP-15 96 - Add a close method with a timeout in the producerper partition metrics for in-sync and assigned replica count | 0.910.02.0 | |||||
245 | KIP-13 - Quota Design94 Session Windows | 0.910.02.0 | |||||
246 | KIP-12 - Kafka Sasl/Kerberos and SSL implementation93: Improve invalid timestamp handling in Kafka Streams | 0.910.02.0 | |||||
247 | KIP-11 - Kafka Authorizer design92 - Add per partition lag metrics to KafkaConsumer | 0.910.02.0 | |||||
248 | KIP-8 - Add a flush method to the producer API91 Provide Intuitive User Timeouts in The Producer | 2.10.9.0.0 | |||||
249 | KIP-4 - Metadata Protocol Changes90 - Remove zkClient dependency from Streams | 0.10.02.0 | |||||
250 | KIP-4 - Command line and centralized administrative operations89: Allow sink connectors to decouple flush and offset commit | 0.910.02.0, | |||||
251 | KIP-88: OffsetFetch Protocol Update | 0.10.2.0 | |||||
252 | KIP-86: Configurable SASL callback handlers | 2.0, 0.10.1.0 | |||||
251253 | KIP-3 - Mirror Maker Enhancement85: Dynamic JAAS configuration for Kafka clients | 0.910.02.0 | |||||
252254 | KIP-2 - Refactor brokers to allow listening on multiple ports and IPs84: Support SASL SCRAM mechanisms | 0.910.02.0 | |||||
253255 | KIP-1 - Remove support of request.required.acks82 - Add Record Headers | 0.911.0.0 |
...
...
KIPs under discussion
Please insert new rows in sorted order (ascending by KIP number). Please move the adopted/accepted KIPs to the "Adopted KIPs" table above.
Dormant/inactive KIPs
Please insert new rows in sorted order (ascending by KIP number).
Discarded KIPs
Please insert new rows in sorted order (ascending by KIP number).
Discarded KIPs
KIP | Comment | ||||||||
---|---|---|---|---|---|---|---|---|---|
KIP-5 - Broker Configuration Management | Superseded by KIP-21 | ||||||||
KIP-7 - Security - IP Filtering | |||||||||
KIP-9 - SSL Support | Adopted via KAFKA-1690 but not via this KIP process | ||||||||
KIP-18 - JBOD Support | Superseded by KIP-112 and KIP-113 | ||||||||
KIP-24 - Remove ISR information from TopicMetadataRequest and add broker level metadata request | |||||||||
KIP-29 - Add an IsrPropagateIntervalMs configuration to KafkaConfig | No longer need after KAFKA-2722 | ||||||||
KIP-34 Add Partitioner Change Listener to Partitioner Interface for Multiple Use Case | Per feedback it is better do this parition() method and avoid Thread Coordination etc. | ||||||||
KIP-80: Kafka Rest Server | |||||||||
KIP-83 - Allow multiple SASL authenticated Java clients in a single JVM process | After KIP-85, no interface changes, reduced to KAFKA-4180 | ||||||||
KIP-116: Add State Store Checkpoint Interval Configuration | Currently not needed as checkpointing can be done on commit interval. | ||||||||
KIP-127: Pluggable JAAS LoginModule configuration for SSL | Similar functionality can be implemented using a custom PrincipalBuilder. | ||||||||
KIP-132: Augment KStream.print to allow extra parameters in the printed string | Duplicated by KIP-160 | ||||||||
KIP-184: Rename LogCleaner and related classes to LogCompactor | A lot of configuration change will cause more trouble and doesn't seems to be worth it | ||||||||
KIP-111: Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker. | Covered by KIP-189 | ||||||||
KIP-147: Add missing type parameters to StateStoreSupplier factories and KGroupedStream/Table methods | Covered by KIP-182 | ||||||||
KIP-165: Extend Interactive Queries for return latest update timestamp per key | Covered by KIP-258 | ||||||||
KIP-170: Enhanced TopicCreatePolicy and introduction of TopicDeletePolicy | Superseded by KIP-201: Rationalising Policy interfaces | ||||||||
KIP 172: Add regular-expression topic support for sink connector | Superseded by KIP-215: Add topic regex support for Connect sinks | KIP-179: Change ReassignPartitionsCommand to use AdminClient | |||||||
KIP 230: Name Windowing Joins | Covered by KIP-372: Naming Repartition Topics for Joins and Grouping | ||||||||
KIP-232: Detect outdated metadata using per-partition leaderEpoch field | Merged into KIP-320: Allow fetchers to detect and handle log truncation | ||||||||
KIP-246: Connect producers and consumers should inherit worker configs | Withdrawn because proposal could not be made backward compatible with existing behavior | ||||||||
KIP-262: Metadata should include number of state stores for task | Rejected since metadata is no longer required. | ||||||||
KIP-263: Allow broker to skip sanity check of inactive segments on broker startup | Withdrawn because the solution we agreed on does not require interface change | ||||||||
KIP-286: producer.send() should not block on metadata update | Withdrawn because the benefit of not having to wait for metadata is probably not worth the complexity added in producer. | ||||||||
KIP-288: [DISCARDED] Consumer.poll() timeout semantic change and new waitForAssignment method | Discarded in deference to KIP-266 | ||||||||
KIP-310: Add a Kafka Source Connector to Kafka Connect | Withdrawn in favor of KIP-382. | ||||||||
KIP-327: Add describe all topics API to AdminClient | Withdrawn in-favor of filter support in Metadata API and KIP-142 | ||||||||
KIP-344: The auto-generated client id should be passed to MetricsReporter | Bug fix so KIP not needed | ||||||||
KIP-364: Remove implicit Materialized, Consumed and Produced | Subsumed by KIP-365 | ||||||||
KIP-404: Add Kafka Connect configuration parameter for disabling WADL output on OPTIONS request | Discarded, reported as a bug by
| ||||||||
KIP-248 - Create New ConfigCommand That Uses The New AdminClient | Discarded by author as it's partly covered in KIP-339 and KIP-422 |
KIP Discussion Recordings
...