This page describes a proposed Kafka Improvement Proposal (KIP) process for proposing a major change to Kafka.
To create your own KIP, click on
Create from template | ||||||||
---|---|---|---|---|---|---|---|---|
|
dev@kafka.apache.org
and request permission (http://kafka.apache.org/contact). Also add an entry to the table KIPs under discussion (for Streams API KIPs, please also add it to Kafka Streams sub page)....
- Click
. Take the next available KIP number and give your proposal a descriptive heading. e.g. "KIP 42: Allow Infinite Retention With Bounded Disk Usage".Create from template templateName 54329345 templateId 54329345 title KIP-NEXT: Insert Title Here buttonLabel Create KIP - Fill in the sections as described above
- Start a [DISCUSS] thread on the Apache mailing list. Please ensure that the subject of the thread is of the format [DISCUSS] KIP-{your KIP number} {your KIP heading} The and the body contains a link to your new KIP. The discussion should happen on the mailing list, not on the wiki, since the wiki comment system doesn't work well for larger discussions. In the process of the discussion you may update the proposal. You should let people know the changes you are making. When you feel you have a finalized proposal
- Once the proposal is finalized call a [VOTE] to have the proposal adopted. These proposals are more serious than code changes and more serious even than release votes. The criteria for acceptance is lazy majority. The vote should remain open for at least 72 hours.
- Please update the KIP wiki page, and the index below, to reflect the current stage of the KIP after a vote. This acts as the permanent record indicating the result of the KIP (e.g., Accepted or Rejected). Also report the result of the KIP vote to the voting thread on the mailing list so the conclusion is clear.
KIP round-up
Next KIP Number: 423530
Use this number as the identifier for your KIP and increment this value.
Adopted KIPs
Please insert new rows in sorted order (descending by KIP number).
KIP (please keep this sorted by KIP number) | Release | |||||
---|---|---|---|---|---|---|
1 | KIP- | 414: Expose Embedded ClientIds in Kafka Streams2.2.0500: Replace ZooKeeper with a Self-Managed Metadata Quorum | ||||
2 | KIP- | 402: Improve fairness in SocketServer processors482: The Kafka Protocol should Support Optional Tagged Fields | 2. | 24.0 (WIP) | ||
3 | KIP-394: Require member.id for initial join group request-504 - Add new Java Authorizer Interface | 2.24.0 (WIP) | ||||
4 | KIP-393: Time windowed serde to properly deserialize changelog input topic503: Add metric for number of topics marked for deletion | 2.24.0 (WIP) | ||||
5 | KIP-386: Standardize on Min/Avg/Max metrics' default value497: Add inter-broker API to alter ISR | 2.24.0 (WIP) | ||||
6 | KIP-382: MirrorMaker 2.0496: Administrative API to delete consumer offsets | 2.24.0 (WIP) | ||||
7 | KIP- | 380: Detect outdated control requests and bounced brokers using broker generation495: Dynamically Adjust Log Levels in Connect | 2. | 24.0 (WIP) | ||
8 | KIP-377: TopicCommand to use AdminClient488: Clean up Sum,Count,Total Metrics | 2.24.0 (WIP) | ||||
9 | KIP-376: Implement AutoClosable on appropriate classes that want to be used in a try-with-resource statement484: Expose metrics for group and transaction metadata loading duration | 2.24.0 (WIP) | ||||
10 | KIP-372: Naming Repartition Topics for Joins and Grouping481: SerDe Improvements for Connect Decimal type in JSON | 2.14.0 (WIP) | ||||
11 | KIP-371: Add a configuration to build custom SSL principal name480: Sticky Partitioner | 2.24.0 (WIP) | ||||
12 | KIP 368: Allow SASL Connections to Periodically Re-Authenticate-479: Add Materialized to Join | 2.24.0(WIP) | ||||
13 | KIP-367 Introduce close(Duration) to Producer and AdminClient instead of close(long, TimeUnit)476: Add Java AdminClient Interface | 2.24.0 | ||||
14 | KIP-366: Make FunctionConversions deprecated475: New Metrics to Measure Number of Tasks on a Connector | 2.14.0 (WIP) | ||||
15 | KIP- | 365: Materialized, Serialized, Joined, Consumed and Produced with implicit Serde474: To deprecate WindowStore#put(key, value) | 2. | 14.0 (WIP) | ||
16 | KIPKIP- | 361: Add Consumer Configuration to Disable Auto Topic Creation471: Expose RocksDB Metrics in Kafka Streams | 2. | 24.0 (WIP) | ||
17 | KIP-359: Verify leader epoch in produce requests467: Augment ProduceResponse error messaging for specific culprit records | 2.24.0 (WIP) | ||||
18 | KIP-358: Migrate Streams API to Duration instead of long ms times465: Add Consolidated Connector Endpoint to Connect REST API | 2.13.0 | ||||
19 | KIP- | 357: Add support to list ACLs per principal464: Defaults for AdminClient#createTopic | 2. | 14.0 | ||
20 | KIP-356: Add withCachingDisabled() to StoreBuilder462: Use local thread id for KStreams | 2.13.0 | ||||
21 | KIP-353461: Improve Kafka Streams Timestamp SynchronizationReplica Fetcher behavior at handling partition failure | 2.13.0 | ||||
22 | KIP- | 351: Add --under-min-isr option to describe topics command460: Admin Leader Election RPC | 2. | 24.0 | (WIP)||
23 | KIP- | 346: Improve LogCleaner behavior on error458: Connector Client Config Override Policy | 2.3. | 10 | ||
24 | KIP-342 Add support for custom SASL extensions in OAuthBearer authentication455: Create an Administrative API for Replica Reassignment | 2.14.0 (WIP) | ||||
25 | KIP-341: Update Sticky Assignor's User Data Protocol454: Expansion of the ConnectClusterState interface | 2.23.0 (WIP) | ||||
26 | KIP-340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file453: Add close() method to RocksDBConfigSetter | 2.3.2.1.0 | ||||
27 | KIP- | 339: Create a new IncrementalAlterConfigs API449: Add connector contexts to log messages in Connect workers | 2. | 23.0 | (WIP)||
28 | KIP-338 Support to exclude the internal topics in kafka-topics.sh command446: Add changelog topic configuration to KTable suppress | 2.14.0 (WIP) | ||||
29 | KIP-336: Consolidate ExtendedSerializer/Serializer and ExtendedDeserializer/Deserializer445: In-memory Session Store | 2.13.0 | ||||
30 | 332 | Update AclCommand to use AdminClient API | 2. | 13.0 | ||
31 | KIP-331 Add default implementation to close() and configure() for Serializer, Deserializer and Serde442: Return to default max poll interval in Streams | 2.23.0 (WIP) | ||||
32 | KIP-330: Add retentionPeriod in SessionBytesStoreSupplier440: Extend Connect Converter to support headers | 2.14.0 (WIP) | ||||
33 | KIP-328: Ability to suppress updates for KTables436: Add a metric indicating start time | 2.23.0 (WIP) | ||||
34 | KIP-324: Add method to get metrics() in AdminClient434: Add Replica Fetcher and Log Cleaner Count Metrics | 2.14.0 (WIP) | ||||
35 | KIP- | 322: Return new error code for DeleteTopics API when topic deletion disabled.430 - Return Authorized Operations in Describe Responses | 2. | 13.0 | ||
36 | KIP-321: Update TopologyDescription to better represent Source and Sink Nodes429: Kafka Consumer Incremental Rebalance Protocol | 2.14.0 (WIP) | ||||
37 | KIP- | 320: Allow fetchers to detect and handle log truncation428: Add in-memory window store | 2. | 13.0 | ||
38 | KIP- | 319: Replace segments with segmentInterval in WindowBytesStoreSupplier427: Add AtMinIsr topic partition category (new metric & TopicCommand option) | 2. | 13.0 | ||
39 | KIP-313: Add KStream.flatTransform and KStream.flatTransformValues 425: Add some Log4J Kafka Appender Properties for Producing to Secured Brokers | 2.23.0 (WIP) | ||||
40 | KIP-312 Add Overloaded StreamsBuilder Build Method to Accept java.util.Properties421: Support resolving externalized secrets in AbstractConfig | 2.13.0 | ||||
41 | KIP-308: Support dynamic update of max.connections.per.ip/max.connections.per.ip.overrides420: Add Single Value Fetch in Session Stores | 2.12.0 | ||||
42 | KIP- | 306: Configuration for Delaying Response to Failed Authentication417: Allow JmxTool to connect to a secured RMI port | 2. | 13.0 | ||
43 | KIP-305: Add Connect primitive number converters415: Incremental Cooperative Rebalancing in Kafka Connect | 2.03.0 | ||||
44 | KIP-303: Add Dynamic Routing in Streams Sink414: Expose Embedded ClientIds in Kafka Streams | 2.02.0 | ||||
45 | KIP- | 302 - Enable Kafka clients to use all DNS resolved IP addresses412: Extend Admin API to support dynamic application log levels | 2. | 14.0 (WIP) | ||
46 | 298: Error Handling in Connect | 2. | 03.0 | |||
47 | KIP-297: Externalizing Secrets for Connect Configurations402: Improve fairness in SocketServer processors | 2.2.0 (partially implemented) / 2.3.0 | ||||
48 | KIP-295 Add Streams Configuration Allowing for Optional Topology Optimization401: TransformerSupplier/ProcessorSupplier StateStore connecting | 2.04.0 (WIP) | ||||
49 | KIP-294 - Enable TLS hostname verification by default396: Add Commit/List Offsets Operations to AdminClient | 2.04.0 (WIP) | ||||
50 | KIP-292: Add transformValues() method to KTable394: Require member.id for initial join group request | 2.02.0 | ||||
51 | KIP-290: Support for Prefixed ACLs393: Time windowed serde to properly deserialize changelog input topic | 2.02.0 | ||||
52 | KIP- | 289: Improve the default group id behavior in KafkaConsumer389: Introduce a configurable consumer group size limit | 2.2.0 | |||
53 | KIP- | 285: Connect Rest Extension Plugin386: Standardize on Min/Avg/Max metrics' default value | 2. | 02.0 | ||
54 | KIP-284: Set default retention ms for Streams repartition topics to Long.MAX_VALUE382: MirrorMaker 2.0 | 2.04.0 | ||||
55 | 283: Efficient Memory Usage for Down-Conversion | 2. | 04.0 (WIP) | |||
56 | KIP-282: Add the listener name to the authentication context380: Detect outdated control requests and bounced brokers using broker generation | 2.02.0 | ||||
57 | KIP-281: ConsumerPerformance: Increase Polling Loop Timeout and Make It Reachable by the End User377: TopicCommand to use AdminClient | 2.02.0 | ||||
58 | KIP- | 279: Fix log divergence between leader and follower after fast leader fail over376: Implement AutoClosable on appropriate classes that want to be used in a try-with-resource statement | 2. | 02.0 | ||
59 | KIP-278 - Add version 374: Add '--help' option to all available Kafka 's CLI commands | 2.02.0 | ||||
60 | KIP-277 - Fine Grained ACL for CreateTopics API372: Naming Repartition Topics for Joins and Grouping | 2.01.0 | ||||
61 | KIP-276 - Add StreamsConfig prefix for different consumers371: Add a configuration to build custom SSL principal name | 2.02.0 | ||||
62 | KIP-274: Kafka Streams Skipped Records Metrics369: Alternative Partitioner to Support "Always Round-Robin" Selection | 2.04.0 | ||||
63 | KIP | -272: Add API version tag to broker's RequestsPerSec metric368: Allow SASL Connections to Periodically Re-Authenticate | 2. | 02.0 | ||
64 | KIP-270 - A Scala Wrapper Library for Kafka Streams367 Introduce close(Duration) to Producer and AdminClient instead of close(long, TimeUnit) | 2.02.0 | ||||
65 | KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade366: Make FunctionConversions deprecated | 2.01.0 | ||||
66 | KIP- | 267: Add Processor Unit Test Support to Kafka Streams Test Utils365: Materialized, Serialized, Joined, Consumed and Produced with implicit Serde | 2. | 01.0 | ||
67 | KIPKIP-266: Fix consumer indefinite blocking behavior361: Add Consumer Configuration to Disable Auto Topic Creation | 2.03.0 | ||||
68 | KIP- | 265: Make Windowed Serde to public APIs360: Improve handling of unknown producer | 2. | 04.0 (WIP) | ||
69 | KIP- | 261: Add Single Value Fetch in Window Stores359: Verify leader epoch in produce requests | 2. | 04.0 (WIP) | ||
70 | KIP-258358: Allow to Store Record Timestamps in RocksDBMigrate Streams API to Duration instead of long ms times | 2.21.0 | ||||
71 | KIP-257 - Configurable Quota Management357: Add support to list ACLs per principal | 2.01.0 | ||||
72 | KIP-255: OAuth Authentication via SASL/OAUTHBEARER356: Add withCachingDisabled() to StoreBuilder | 2.01.0 | ||||
73 | KIP- | 251: Allow timestamp manipulation in Processor API354: Add a Maximum Log Compaction Lag | 2. | 03.0 | ||
74 | KIP- | 249: Add Delegation Token Operations to KafkaAdminClient353: Improve Kafka Streams Timestamp Synchronization | 2. | 01.0 | ||
75 | KIP- | 247: Add public test utils for Kafka Streams352: Distinguish URPs caused by reassignment | 2.4.0 (WIP) | 1.1.0|||
76 | KIP- | 245: Use Properties instead of StreamsConfig in KafkaStreams constructor351: Add --under-min-isr option to describe topics command | 2. | 03.0 | ||
77 | KIP-244: Add Record Header support to Kafka Streams Processor API346: Improve LogCleaner behavior on error | 2.0.01 | ||||
78 | KIP- | 243: Make ProducerConfig and ConsumerConfig constructors public345: Introduce static membership protocol to reduce consumer rebalances | 2.4.0 (WIP), partially available in 2.3 | 1.1.0 | ||
79 | KIP- | 239 Add queryableStoreName() to GlobalKTable342 Add support for custom SASL extensions in OAuthBearer authentication | 2 | 1.1.0 | ||
80 | KIP- | 238: Expose Kafka cluster ID in Connect REST API341: Update Sticky Assignor's User Data Protocol | 2.3 | 1.1.0 | ||
81 | KIP- | 237: More Controller Health Metrics340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file | 2. | 01.0 | ||
82 | KIP-235: Add DNS alias support for secured connection339: Create a new IncrementalAlterConfigs API | 2.13.00 | ||||
83 | KIP | -233: Simplify StreamsBuilder#addGlobalStore-338 Support to exclude the internal topics in kafka-topics.sh command | 2 | 1.1.0 | ||
84 | KIP-231: Improve the Required ACL of ListGroups API336: Consolidate ExtendedSerializer/Serializer and ExtendedDeserializer/Deserializer | 2.1.0 | ||||
85 | KIP-229: DeleteGroups 332: Update AclCommand to use AdminClient API | 12.1.0 | ||||
86 | KIP- | 227 - Introduce Incremental FetchRequests to Increase Partition Scalability331 Add default implementation to close() and configure() for Serializer, Deserializer and Serde | 2.3. | 1.1.0 | ||
87 | KIP- | 226 - Dynamic Broker Configuration330: Add retentionPeriod in SessionBytesStoreSupplier | 2 | 1.1.0 | ||
88 | KIP- | 225 - Use tags for consumer “records.lag” metrics328: Ability to suppress updates for KTables | 2.1.0 (partially implemented) / 2.3.0 (WIP) | 1.1.0|||
89 | KIP- | 224324: Add | configuration parameter `retries` to Streams APImethod to get metrics() in AdminClient | 2 | 1.1.0 | |
90 | KIP-223 - Add per-topic min lead and per-partition lead metrics to KafkaConsumer322: Return new error code for DeleteTopics API when topic deletion disabled. | 2.01.0 | ||||
91 | KIP- | 222 - Add Consumer Group operations to Admin API321: Update TopologyDescription to better represent Source and Sink Nodes | 2. | 01.0 | ||
92 | KIP- | 220: Add AdminClient into Kafka Streams' ClientSupplier320: Allow fetchers to detect and handle log truncation | 2.1.0 (partially implemented) / 2.2 | 1.1.0 | ||
93 | KIP- | 219 - Improve quota communication319: Replace segments with segmentInterval in WindowBytesStoreSupplier | 2. | 01.0 | ||
94 | KIP- | 218: Make KafkaFuture.Function java 8 lambda compatible313: Add KStream.flatTransform and KStream.flatTransformValues | 2.2.0 (partially implemented) / 2.3 | 1.1.0 | ||
95 | KIP- | 215: Add topic regex support for Connect sinks312 Add Overloaded StreamsBuilder Build Method to Accept java.util.Properties | 2 | 1.1.0 | ||
96 | KIP- | 214309: Add | zookeeper.max.in.flight.requests config to the broker1.1toUpperCase support to sasl.kerberos.principal.to.local rule | 2.4.0 | ||
97 | KIP-212: Enforce set of legal characters for connector names308: Support dynamic update of max.connections.per.ip/max.connections.per.ip.overrides | 21.1.0 | ||||
98 | KIP- | 211: Revise Expiration Semantics of Consumer Group Offsets307: Allow to define custom processor names with KStreams DSL | 2. | 13.0 (partial) | ||
99 | KIP- | 210 - Provide for custom error handling when Kafka Streams fails to produce306: Configuration for Delaying Response to Failed Authentication | 2 | 1.1.0 | ||
100 | KIP-208305: Add SSL support to Kafka Connect REST interfaceConnect primitive number converters | 2.01.1.0 | ||||
101 | KIP- | 207: Offsets returned by ListOffsetsResponse should be monotonically increasing even during a partition leader change303: Add Dynamic Routing in Streams Sink | 2. | 20.0 | ||
102 | KIP-206: Add support for UUID serialization and deserialization302 - Enable Kafka clients to use all DNS resolved IP addresses | 2.1.0 | ||||
103 | KIP- | 205300: Add | all() and range() API to ReadOnlyWindowStoreWindowed KTable API in StreamsBuilder | 2.4 | 1.1.0 | |
104 | KIP- | 204 : Adding records deletion operation to the new Admin Client API298: Error Handling in Connect | 2.0 | 1.1.0 | ||
105 | KIP- | 203: Add toLowerCase support to sasl.kerberos.principal.to.local rule297: Externalizing Secrets for Connect Configurations | 2.0 | 1.1.0 | ||
106 | KIP- | 202: Move merge() from StreamsBuilder to KStream295 Add Streams Configuration Allowing for Optional Topology Optimization | 2 | 1.0.0 | ||
107 | KIP | -198: Remove ZK dependency from Streams Reset Tool-294 - Enable TLS hostname verification by default | 2 | 1.0.0 | ||
108 | 197 Connect REST API should include the connector type when describing a connector | Accepted1.0.0 | ||||
109 | KIP- | 196292: Add | metrics to Kafka Connect frameworktransformValues() method to KTable | 2 | 1.0.0 | |
110 | KIP- | 195: AdminClient.createPartitions290: Support for Prefixed ACLs | 2 | 1.0.0 | ||
111 | KIP- | 192 : Provide cleaner semantics when idempotence is enabled289: Improve the default group id behavior in KafkaConsumer | 2.2 | 1.0.0 | ||
112 | KIP-191: KafkaConsumer.subscribe() overload that takes just Pattern | 1.0.0 | 285: Connect Rest Extension Plugin | 2.0.0 | ||
113 | KIP- | 190: Handle client-ids consistently between clients and brokers284: Set default retention ms for Streams repartition topics to Long.MAX_VALUE | 2 | 1.0.0 | ||
114 | KIP- | 189: Improve principal builder interface and add support for SASL283: Efficient Memory Usage for Down-Conversion | 2 | 1.0.0 | ||
115 | KIP- | 188 - Add new metrics to support health checks282: Add the listener name to the authentication context | 2 | 1.0.0 | ||
116 | KIP- | 187 - Add cumulative count metric for all Kafka rate metrics281: ConsumerPerformance: Increase Polling Loop Timeout and Make It Reachable by the End User | 2 | 1.0.0 | ||
117 | KIP-186: Increase offsets retention default to 7 days279: Fix log divergence between leader and follower after fast leader fail over | 2.0.0 | ||||
118 | KIP- | 183 - Change PreferredReplicaLeaderElectionCommand to use AdminClient278 - Add version option to Kafka's commands | 2.0.0 | |||
119 | KIP | -182: Reduce Streams DSL overloads and allow easier use of custom storage engines-277 - Fine Grained ACL for CreateTopics API | 2 | 1.0.0 | ||
120 | KIP-276 - 180: Add a broker metric specifying the number of consumer group rebalances in progressAdd StreamsConfig prefix for different consumers | 2.01.1.0 | ||||
121 | KIP-177: Consumer perf tool should count rebalance time274: Kafka Streams Skipped Records Metrics | 21.0.0 | ||||
122 | KIP-176: Remove deprecated new-consumer option for tools272: Add API version tag to broker's RequestsPerSec metric | 2.0.0 | ||||
123 | KIP- | 175: Additional '--describe' views for ConsumerGroupCommand270 - A Scala Wrapper Library for Kafka Streams | 2.0 | 1.1.0 | ||
124 | KIP-174 - Deprecate and remove internal converter configs in WorkerConfig268: Simplify Kafka Streams Rebalance Metadata Upgrade | 2.0.0 | ||||
125 | KIP-173267: Add prefix to StreamsConfig to enable setting default internal topic configsProcessor Unit Test Support to Kafka Streams Test Utils | 21.0.0 | ||||
126 | KIP- | 171 - Extend Consumer Group Reset Offset for Stream Application266: Fix consumer indefinite blocking behavior | 2.0 | 1.1.0 | ||
127 | KIP-168: Add GlobalTopicCount and GlobalPartitionCount metric per cluster265: Make Windowed Serde to public APIs | 21.0.0 | ||||
128 | KIP-167261: Add interface for the state store restoration processSingle Value Fetch in Window Stores | 21.0.0 | ||||
129 | KIP- | 164 - Add UnderMinIsrPartitionCount and per-partition UnderMinIsr metrics258: Allow to Store Record Timestamps in RocksDB | 2.3.0 (partially implemented) | 1.0.0|||
130 | KIP- | 163: Lower the Minimum Required ACL Permission of OffsetFetch257 - Configurable Quota Management | 2 | 1.0.0 | ||
131 | KIP- | 162: Enable topic deletion by default255: OAuth Authentication via SASL/OAUTHBEARER | 2 | 1.0.0 | ||
132 | KIP-161: streams deserialization exception handlers251: Allow timestamp manipulation in Processor API | 21.0.0 | ||||
133 | KIP-160: Augment KStream.print(), KStream.writeAsText() to allow users pass in extra parameters in the printed string249: Add Delegation Token Operations to KafkaAdminClient | 21.0.0 | ||||
134 | KIP- | 157 - Add consumer config options to streams reset tool247: Add public test utils for Kafka Streams | 1. | 01.0 | ||
135 | KIP- | 156 Add option "dry run" to Streams application reset tool245: Use Properties instead of StreamsConfig in KafkaStreams constructor | 2 | 0.11.0.0 | ||
136 | KIP-155 - Add range scan for windowed state stores244: Add Record Header support to Kafka Streams Processor API | 2.0.11.0.0 | ||||
137 | KIP-154 Add Kafka Connect configuration properties for creating internal topics243: Make ProducerConfig and ConsumerConfig constructors public | 1.1.00.11.0.0 | ||||
138 | KIP-153: Include only client traffic in BytesOutPerSec metric239 Add queryableStoreName() to GlobalKTable | 1.10.11.0.0 | ||||
139 | KIP-152 - Improve diagnostics for SASL authentication failures238: Expose Kafka cluster ID in Connect REST API | 1.01.0 | ||||
140 | KIP-151 Expose Connector type in REST API237: More Controller Health Metrics | 2.0.11.0.0 | ||||
141 | KIP- | 150 - Kafka-Streams Cogroup235: Add DNS alias support for secured connection | 2. | 2.0 (WIP)1.0 | ||
142 | KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner233: Simplify StreamsBuilder#addGlobalStore | 1.1.0 (partially implemented) (WIP for 2.2.0) | ||||
143 | KIP-146 - Classloading Isolation in Connect | 0.11.0.0 | ||||
143 | KIP-231: Improve the Required ACL of ListGroups API | 2.1.0 | ||||
144 | KIP-229: DeleteGroups API | 1.1.0 | ||||
145 | KIP-227 - Introduce Incremental FetchRequests to Increase Partition Scalability | 144 | KIP-145 - Expose Record Headers in Kafka Connect | 1.1.0 | ||
145146 | KIP-144: Exponential backoff for broker reconnect attempts | 0.11.0.0 | 146 | KIP-143: Controller Health Metrics | 226 - Dynamic Broker Configuration | 1.1.00.11.0.0 |
147 | KIP-140: Add administrative RPCs for adding, deleting, and listing ACLs-225 - Use tags for consumer “records.lag” metrics | 1.10.11.0.0 | ||||
148 | KIP-138: Change punctuate semantics224: Add configuration parameter `retries` to Streams API | 1.01.0 | ||||
149 | KIP-137: Enhance TopicCommand --describe to show topics marked for deletion223 - Add per-topic min lead and per-partition lead metrics to KafkaConsumer | 20.11.0.0 | ||||
150 | KIP-136: Add Listener name to SelectorMetrics tags222 - Add Consumer Group operations to Admin API | 20.11.0.0 | ||||
151 | KIP-134: Delay initial consumer group rebalance221: Enhance DSL with Connecting Topic Creation and Repartition Hint | 2.4.0 (WIP)0.11.0.0 | ||||
152 | KIP-133: Describe and Alter Configs Admin APIs220: Add AdminClient into Kafka Streams' ClientSupplier | 1.10.11.0.0 | ||||
153 | KIP | -130: Expose states of active tasks to KafkaStreams public API-219 - Improve quota communication | 2 | 1.0.0 | ||
154 | KIP-129: Streams Exactly-Once Semantics218: Make KafkaFuture.Function java 8 lambda compatible | 1.1.00.11.0.0 | ||||
155 | KIP-128215: Add ByteArrayConverter topic regex support for Kafka Connect sinks | 01.111.0.0 | ||||
156 | KIP-126 - Allow KafkaProducer to split and resend oversized batches.214: Add zookeeper.max.in.flight.requests config to the broker | 1.10.11.0.0 | ||||
157 | KIP- | 124 - Request rate quotas213 Support non-key joining in KTable | 2.4.0 (WIP) | 0.11.0.0|||
158 | KIP-123: Allow per stream/table timestamp extractor212: Enforce set of legal characters for connector names | 1.10.11.0.0 | ||||
159 | KIP-122: Add Reset 211: Revise Expiration Semantics of Consumer Group Offsets tooling | 02.11.01.0 | ||||
160 | KIP-121: Add KStream peek method-210 - Provide for custom error handling when Kafka Streams fails to produce | 1.1.00.11.0.0 | ||||
161 | KIP-120: Cleanup Kafka Streams builder API208: Add SSL support to Kafka Connect REST interface | 1.01.0 | ||||
162 | KIP-207: Offsets returned by ListOffsetsResponse should be monotonically increasing even during a partition leader change | 2.2119: Drop Support for Scala 2.10 in Kafka 0.110.11.0.0 | ||||
163 | KIP- | 118206: | Drop Support for Java 7Add support for UUID serialization and deserialization | 2. | 01.0 | |
164 | KIP-117205: Add a public AdminClient API for Kafka admin operationsall() and range() API to ReadOnlyWindowStore | 1.1.00.11.0.0 | ||||
165 | KIP-204 : Adding records deletion operation to the new Admin Client API | 1.1.0115: Enforce offsets.topic.replication.factor upon __consumer_offsets auto topic creation0.11.0.0 | ||||
166 | KIP-114: KTable state stores and improved semantics203: Add toLowerCase support to sasl.kerberos.principal.to.local rule | 1.1.00.11.0.0 | ||||
167 | KIP-113: Support replicas movement between log directories202: Move merge() from StreamsBuilder to KStream | 1.10.0 | ||||
168 | KIP-112: Handle disk failure for JBOD198: Remove ZK dependency from Streams Reset Tool | 1.0.0 | ||||
169 | KIP- | 110: Add Codec for ZStandard Compression197 Connect REST API should include the connector type when describing a connector | 1.0 | 2.1.0 | ||
170 | KIP-109: Old Consumer Deprecation196: Add metrics to Kafka Connect framework | 1.0.11.0.0 | ||||
171 | KIP-108: Create Topic Policy195: AdminClient.createPartitions | 1.0.0.10.2.0 | ||||
172 | KIP-107: Add deleteRecordsBefore() API in AdminClient192 : Provide cleaner semantics when idempotence is enabled | 10.11.0.0 | ||||
173 | KIP-191: KafkaConsumer.subscribe() overload that takes just Pattern | 1.106 - Change Default unclean.leader.election.enabled from True to False0.11.0.0 | ||||
174 | KIP-105: Addition of Record Level for Sensors190: Handle client-ids consistently between clients and brokers | 1.0.0.10.2.0 | ||||
175 | KIP-104: Granular Sensors for Streams 189: Improve principal builder interface and add support for SASL | 1.0.0.10.2.0 | ||||
176 | KIP-103: Separation of Internal and External traffic-188 - Add new metrics to support health checks | 1.0.0.10.2.0 | ||||
177 | KIP-102 187 - Add close with timeout for consumerscumulative count metric for all Kafka rate metrics | 1.0.0.10.2.0 | ||||
178 | KIP-101 - Alter Replication Protocol to use Leader Epoch rather than High Watermark for Truncation186: Increase offsets retention default to 7 days | 20.11.0.0 | ||||
179 | KIP-100 - Relax Type constraints in Kafka Streams API183 - Change PreferredReplicaLeaderElectionCommand to use AdminClient | 20.10.2.0 | ||||
180 | KIP-99: Add Global Tables to Kafka Streams182: Reduce Streams DSL overloads and allow easier use of custom storage engines | 1.0.0.10.2.0 | ||||
181 | KIP-98 - Exactly Once Delivery and Transactional Messaging180: Add a broker metric specifying the number of consumer group rebalances in progress | 1.1.00.11.0.0 | ||||
182 | KIP-97: Improved Kafka Client RPC Compatibility Policy177: Consumer perf tool should count rebalance time | 1.00.10.2.0 | ||||
183 | KIP- | 96 - Add per partition metrics for in-sync and assigned replica count176: Remove deprecated new-consumer option for tools | 2.0. | 0.10.2.0 | ||
184 | KIP-94 Session Windows-175: Additional '--describe' views for ConsumerGroupCommand | 1.10.10.2.0 | ||||
185 | KIP-93: Improve invalid timestamp handling in Kafka Streams174 - Deprecate and remove internal converter configs in WorkerConfig | 2.0.0.10.2.0 | ||||
186 | KIP-92 - Add per partition lag metrics to KafkaConsumer173: Add prefix to StreamsConfig to enable setting default internal topic configs | 1.0.00.10.2.0 | ||||
187 | KIP-91 Provide Intuitive User Timeouts in The Producer-171 - Extend Consumer Group Reset Offset for Stream Application | 12.1.0 | ||||
188 | KIP-90 - Remove zkClient dependency from Streams168: Add GlobalTopicCount and GlobalPartitionCount metric per cluster | 1.00.10.2.0 | ||||
189 | KIP-89: Allow sink connectors to decouple flush and offset commit167: Add interface for the state store restoration process | 1.0.0.10.2.0 | ||||
190 | 88: OffsetFetch Protocol Update 164 - Add UnderMinIsrPartitionCount and per-partition UnderMinIsr metrics | 1.0.0.10.2.0 | ||||
191 | 86: Configurable SASL callback handlers | 1 | 2.0.0 | |||
192 | KIP-85: Dynamic JAAS configuration for Kafka clients162: Enable topic deletion by default | 1.0.0.10.2.0 | ||||
193 | KIP-84: Support SASL SCRAM mechanisms161: streams deserialization exception handlers | 1.0.0.10.2.0 | ||||
194 | KIP-82 - Add Record Headers160: Augment KStream.print(), KStream.writeAsText() to allow users pass in extra parameters in the printed string | 10.11.0.0 | ||||
195 | KIP | -81: Bound Fetch memory usage in the consumer-157 - Add consumer config options to streams reset tool | 1.0.0 | 2.2.0 (WIP)|||
196 | KIP- | 79 - ListOffsetRequest/ListOffsetResponse v1 and add timestamp search methods to the new consumer156 Add option "dry run" to Streams application reset tool | 0. | 1011. | 10.0 | |
197 | KIP | -78: Cluster Id-155 - Add range scan for windowed state stores | 0. | 1011. | 10.0 | |
198 | KIP-77: Improve Kafka Streams Join Semantics154 Add Kafka Connect configuration properties for creating internal topics | 0.1011.20.0 | ||||
199 | KIP-75 - Add per-connector Converters153: Include only client traffic in BytesOutPerSec metric | 0.1011.10.0 | ||||
200 | KIP-74: Add Fetch Response Size Limit in Bytes-152 - Improve diagnostics for SASL authentication failures | 1.00.10.1.0 | ||||
201 | KIP-73: Replication Quotas151 Expose Connector type in REST API | 0.1011.10.0 | ||||
202 | KIP-72: Allow putting a bound on memory consumed by Incoming request -150 - Kafka-Streams Cogroup | 2.4.0 (WIP)1.0.0 | ||||
203 | KIP-71: Enable log compaction and deletion to co-exist149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner | 1.1.0 (partially implemented) (WIP for 2.4.0)0.10.1.0 | ||||
204 | KIP-70: Revise Partition Assignment Semantics on New Consumer's Subscription Change-146 - Classloading Isolation in Connect | 0. 1011. 10.0 | ||||
205 | KIP-67: Queryable state for Kafka Streams-145 - Expose Record Headers in Kafka Connect | 1.0.10.1.0 | ||||
206 | KIP-66: Single Message Transforms for Kafka Connect144: Exponential backoff for broker reconnect attempts | 0.10.2.0 / 0.11.0.0 | ||||
207 | KIP-65: Expose timestamps to Connect143: Controller Health Metrics | 0.1011.10.0 | ||||
208 | KIP-63: Unify store and downstream caching in streams140: Add administrative RPCs for adding, deleting, and listing ACLs | 0.1011.10.0 | ||||
209 | KIP-62: Allow consumer to send heartbeats from a background thread138: Change punctuate semantics | 1.00.10.1.0 | ||||
210 | KIP-137: Enhance TopicCommand -60 - Make Java client classloading more flexible-describe to show topics marked for deletion | 0.1011.10.0 | ||||
211 | KIP-58 - Make Log Compaction Point Configurable136: Add Listener name to SelectorMetrics tags | 0.1011.10.0 | ||||
212 | KIP-57 - Interoperable LZ4 Framing134: Delay initial consumer group rebalance | 0.1011.0.0 | ||||
213 | KIP-56: Allow cross origin HTTP requests on all HTTP methods133: Describe and Alter Configs Admin APIs | 0.1011.0.0 | ||||
214 | KIP- | 55: Secure Quotas for Authenticated Users130: Expose states of active tasks to KafkaStreams public API | 1.0. | 0.10.1.0 | ||
215 | KIP-54: Sticky Partition Assignment Strategy129: Streams Exactly-Once Semantics | 0.11.0.0 | ||||
216 | KIP-52: Connector Control APIs128: Add ByteArrayConverter for Kafka Connect | 0.1011.0.0 | ||||
217 | KIP-51 - List Connectors REST API126 - Allow KafkaProducer to split and resend oversized batches. | 0.1011.0.0 | ||||
218 | KIP-50 - Move Authorizer to o.a.k.common package124 - Request rate quotas | 0.1011.10.0 | ||||
219 | KIP-48 Delegation token support for Kafka123: Allow per stream/table timestamp extractor | 0.11.0.01.1.0 | ||||
220 | KIP-45 - Standardize all client sequence interaction on j.u.Collection.122: Add Reset Consumer Group Offsets tooling | 0.1011.0.0 | ||||
221 | KIP-43: Kafka SASL enhancements121: Add KStream peek method | 0. 1011.0.0 | ||||
222 | KIP-42: Add Producer and Consumer Interceptors120: Cleanup Kafka Streams builder API | 10.10.0.0 | ||||
223 | KIP-41: Consumer Max Records119: Drop Support for Scala 2.10 in Kafka 0.11 | 0.1011.0.0 | ||||
224 | KIP-40: ListGroups and DescribeGroup118: Drop Support for Java 7 | 2.0.9.0.0 | ||||
225 | KIP-38: ZooKeeper Authentication117: Add a public AdminClient API for Kafka admin operations | 0.911.0.0 | ||||
226 | KIP-36 - Rack aware replica assignment115: Enforce offsets.topic.replication.factor upon __consumer_offsets auto topic creation | 0.110.10.0.0 | ||||
227 | KIP-35 - Retrieving protocol version114: KTable state stores and improved semantics | 0.1011.0.0 | ||||
228 | KIP-33 - Add a time based log index113: Support replicas movement between log directories | 10.10.1.0 | ||||
229 | KIP-32 - Add timestamps to Kafka message112: Handle disk failure for JBOD | 10.10.0.0 | ||||
230 | KIP- | 31 - Move to relative offsets in compressed message sets110: Add Codec for ZStandard Compression | 2.1.0 | 0.10.0.0|||
231 | KIP-28 - Add a processor client109: Old Consumer Deprecation | 0.1011.0.0 | ||||
232 | KIP-26 - Add Kafka Connect framework for data import/export108: Create Topic Policy | 0.910.02.0 | ||||
233 | KIP-25 - System test improvements107: Add deleteRecordsBefore() API in AdminClient | 0.911.0.0 | ||||
234 | 22 - Expose a Partitioner interface in the new producer 106 - Change Default unclean.leader.election.enabled from True to False | 0.911.0.0 | ||||
235 | KIP-21 - Dynamic Configuration105: Addition of Record Level for Sensors | 0.910.02.0 (WIP) | ||||
236 | KIP-20 Enable log preallocate to improve consume performance under windows and some old Linux file system104: Granular Sensors for Streams | 0.910.02.0 | ||||
237 | KIP-19 - Add a request timeout to NetworkClient103: Separation of Internal and External traffic | 0.910.02.0 | ||||
238 | KIP-16 - Automated Replica Lag Tuning102 - Add close with timeout for consumers | 0.910.02.0 | ||||
239 | KIP-15 - Add a close method with a timeout in the producer101 - Alter Replication Protocol to use Leader Epoch rather than High Watermark for Truncation | 0.911.0.0 | ||||
240 | KIP-13 - Quota Design100 - Relax Type constraints in Kafka Streams API | 0.910.02.0 | ||||
241 | KIP-12 - Kafka Sasl/Kerberos and SSL implementation99: Add Global Tables to Kafka Streams | 0.910.02.0 | ||||
242 | KIP-11 - Kafka Authorizer design98 - Exactly Once Delivery and Transactional Messaging | 0.911.0.0 | ||||
243 | KIP-8 - Add a flush method to the producer API97: Improved Kafka Client RPC Compatibility Policy | 0.910.02.0 | ||||
244 | KIP-4 - Metadata Protocol Changes-96 - Add per partition metrics for in-sync and assigned replica count | 0.10.02.0 | ||||
245 | KIP-4 - Command line and centralized administrative operations94 Session Windows | 0.9.0.0, 0.10.0.0, 0.10.12.0 | ||||
246 | KIP-3 - Mirror Maker Enhancement93: Improve invalid timestamp handling in Kafka Streams | 0.910.02.0 | ||||
247 | KIP-2 - Refactor brokers to allow listening on multiple ports and IPs92 - Add per partition lag metrics to KafkaConsumer | 0.910.02.0 | ||||
248 | KIP-1 - Remove support of request.required.acks | 0.9.0.0 |
...
...
KIP-185: Make exactly once in order delivery the default producer setting
...
...
Discussion
...
...
Voting in progress (restarted 18th January 2019, due to no votes in first attempt)
...
Discussion
...
...
Draft
...
...
...
...
KIPs under discussion
Please insert new rows in sorted order (ascending by KIP number). Please move the adopted/accepted KIPs to the "Adopted KIPs" table above.
Dormant/inactive KIPs
Please insert new rows in sorted order (ascending by KIP number).
Discarded KIPs
Please insert new rows in sorted order (ascending by KIP number).
...
Discarded KIPs
KIP | Comment | |
---|---|---|
KIP-5 - Broker Configuration Management | Superseded by KIP-21 | |
KIP-7 - Security - IP Filtering | ||
KIP-9 - SSL Support | Adopted via KAFKA-1690 but not via this KIP process | |
KIP-18 - JBOD Support | Superseded by KIP-112 and KIP-113 | |
KIP-24 - Remove ISR information from TopicMetadataRequest and add broker level metadata request | ||
KIP-29 - Add an IsrPropagateIntervalMs configuration to KafkaConfig | No longer need after KAFKA-2722 | |
KIP-34 Add Partitioner Change Listener to Partitioner Interface for Multiple Use Case | Per feedback it is better do this parition() method and avoid Thread Coordination etc. | |
KIP-80: Kafka Rest Server | ||
KIP-83 - Allow multiple SASL authenticated Java clients in a single JVM process | After KIP-85, no interface changes, reduced to KAFKA-4180 | |
KIP-116: Add State Store Checkpoint Interval Configuration | Currently not needed as checkpointing can be done on commit interval. | |
KIP-127: Pluggable JAAS LoginModule configuration for SSL | Similar functionality can be implemented using a custom PrincipalBuilder. | |
KIP-132: Augment KStream.print to allow extra parameters in the printed string | Duplicated by KIP-160 | |
KIP-184: Rename LogCleaner and related classes to LogCompactor | A lot of configuration change will cause more trouble and doesn't seems to be worth it | |
KIP-111: Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker. | Covered by KIP-189 | |
KIP-147: Add missing type parameters to StateStoreSupplier factories and KGroupedStream/Table methods | Covered by KIP-182 | |
KIP-165: Extend Interactive Queries for return latest update timestamp per key | Covered by KIP-258 | |
KIP-170: Enhanced TopicCreatePolicy and introduction of TopicDeletePolicy | Superseded by KIP-201: Rationalising Policy interfaces | |
KIP 172: Add regular-expression topic support for sink connector | Superseded by KIP-215: Add topic regex support for Connect sinks | KIP-179: Change ReassignPartitionsCommand to use AdminClient |
KIP 230: Name Windowing Joins | Covered by KIP-372: Naming Repartition Topics for Joins and Grouping | |
KIP-232: Detect outdated metadata using per-partition leaderEpoch field | Merged into KIP-320: Allow fetchers to detect and handle log truncation | |
KIP-246: Connect producers and consumers should inherit worker configs | Withdrawn because proposal could not be made backward compatible with existing behavior | |
KIP-262: Metadata should include number of state stores for task | Rejected since metadata is no longer required. | |
KIP-263: Allow broker to skip sanity check of inactive segments on broker startup | Withdrawn because the solution we agreed on does not require interface change | |
KIP-286: producer.send() should not block on metadata update | Withdrawn because the benefit of not having to wait for metadata is probably not worth the complexity added in producer. | |
KIP-288: [DISCARDED] Consumer.poll() timeout semantic change and new waitForAssignment method | Discarded in deference to KIP-266 | |
KIP-310: Add a Kafka Source Connector to Kafka Connect | Withdrawn in favor of KIP-382. | |
KIP-327: Add describe all topics API to AdminClient | Withdrawn in-favor of filter support in Metadata API and KIP-142 | |
KIP-344: The auto-generated client id should be passed to MetricsReporter | Bug fix so KIP not needed | |
KIP-364: Remove implicit Materialized, Consumed and Produced | Subsumed by KIP-365 | |
KIP-404: Add Kafka Connect configuration parameter for disabling WADL output on OPTIONS request | Discarded, reported as a bug by Jira | |
server | ASF JIRA | |
serverId | 5aa69414-a9e9-3523-82ec-879b028fb15b | key | KAFKA-7759
KIP Discussion Recordings
...