Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

This page describes a proposed Kafka Improvement Proposal (KIP) process for proposing a major change to Kafka.

...

Create from template
templateName54329345
templateId54329345
titleKIP-NEXT: Insert Title Here
buttonLabelCreate KIP

...

Table of Contents

Getting Started

If this is your first time contributing:

  1. Sign up for the Developer mailing list dev@kafka.apache.org . The instructions to sign up are here: http://kafka.apache.org/contact

...

  1. Create a wiki ID (https://cwiki.apache.org/confluence/signup.action)
  2. Create a Jira ID (It's a different system than the wiki) using the ASF Self-serve Portal (https://selfserve.apache.org/jira-account.html)
  3. Send an email to the dev mailing list (dev@kafka.apache.org) containing your wiki ID and Jira ID requesting permissions to contribute to Apache Kafka.

Table of Contents

Purpose

We want to make Kafka a core architectural component for users. We also support a large number of integrations with other tools, systems, and clients. Keeping this kind of usage health requires a high level of compatibility between releases — core architectural elements can't break compatibility or shift functionality from release to release. As a result each new major feature or public api has to be done in a way that we can stick with it going forward.

...

  • Binary log format
  • The network protocol and api behavior
  • Any class for which the build generates Javadoc.  This includes (but is not limited to; look for javadoc {} sections in the public packages under clients build.gradle for the complete list):
    • org/apache/kafka/common/serialization

    • org/apache/kafka/common

    • org/apache/kafka/common/errors

    • org/apache/kafka/clients/producer

    • org/apache/kafka/clients/consumer (eventually, once stable)

  • Configuration, especially client configuration
  • Monitoring
  • Command line tools and arguments

...

  1. Click
    Create from template
    templateName54329345
    templateId54329345
    titleKIP-NEXT: Insert Title Here
    buttonLabelCreate KIP
    . Take the next available KIP number and give your proposal a descriptive heading. e.g. "KIP 42: Allow Infinite Retention With Bounded Disk Usage".
  2. Update the next available KIP number below.
  3. Fill in the sections as described above
  4. Also add an entry to the table KIPs under discussion (for Streams API KIPs, please also add it to Kafka Streams sub page).
  5. Start a [DISCUSS] thread on the Apache mailing list. Start a [DISCUSS] thread on the Apache mailing list. Please ensure that the subject of the thread is of the format [DISCUSS] KIP-{your KIP number} {your KIP heading} and the body contains a link to your new KIP. The discussion should happen on the mailing list, not on the wiki, since the wiki comment system doesn't work well for larger discussions. In the process of the discussion you may update the proposal. You should let people know the changes you are making. When you feel you have a finalized proposal 
  6. Once the proposal is finalized call a [VOTE] to have the proposal adopted. These proposals are more serious than code changes and more serious even than release votes. The criteria for acceptance is lazy majority. The vote should remain open for at least 72 hours.
  7. Please update the KIP wiki page, and the index below, to reflect the current stage of the KIP after a vote. This acts as the permanent record indicating the result of the KIP (e.g., Accepted or Rejected). Also report the result of the KIP vote to the voting thread on the mailing list so the conclusion is clear.e

KIP round-up

Next KIP Number: 4811033

Use this number as the identifier for your KIP and increment this value.

...

Please insert new rows in sorted order (descending by KIP number).

KIP (please keep this sorted by KIP number)

Release

1
KIP-
465: Add Consolidated Connector Endpoint to Connect REST API2.3.02KIP-464: Defaults for AdminClient#createTopic2.4.03KIP-462: Use local thread id for KStreams2.3.04KIP-461: Improve Replica Fetcher behavior at handling partition failure2.3.05KIP-460: Admin Leader Election RPC2.4.06KIP-458: Connector Client Config Override Policy2.3.07KIP-454: Expansion of the ConnectClusterState interface2.3.08KIP-453: Add close() method to RocksDBConfigSetter2.3.09KIP-449: Add connector contexts to log messages in Connect workers 2.3.010KIP-446: Add changelog topic configuration to KTable suppress2.4.0 (WIP)11KIP-445: In-memory Session Store2.3.012

KIP-443: Return to default segment.ms and segment.index.bytes in Streams repartition topics

2.3.013KIP-442: Return to default max poll interval in Streams2.3.014KIP-436: Add a metric indicating start time2.3.015KIP-430 - Return Authorized Operations in Describe Responses2.3.016KIP-429: Kafka Consumer Incremental Rebalance Protocol2.4.0 (WIP)17KIP-428: Add in-memory window store2.3.018KIP-427: Add AtMinIsr topic partition category (new metric & TopicCommand option)2.3.019KIP-425: Add some Log4J Kafka Appender Properties for Producing to Secured Brokers2.3.020KIP-421: Support resolving externalized secrets in AbstractConfig2.3.021KIP-420: Add Single Value Fetch in Session Stores2.2.022KIP-417: Allow JmxTool to connect to a secured RMI port2.3.023KIP-415: Incremental Cooperative Rebalancing in Kafka Connect2.3.024KIP-414: Expose Embedded ClientIds in Kafka Streams2.2.025KIP-412: Extend Admin API to support dvynamic application log levels2.4.0 (WIP)26

KIP-411: Make default Kafka Connect worker task client IDs distinct

2.3.027KIP-402: Improve fairness in SocketServer processors2.2.0 (partially implemented) / 2.3.028KIP-394: Require member.id for initial join group request2.2.0 29KIP-393: Time windowed serde to properly deserialize changelog input topic2.2.030KIP-386: Standardize on Min/Avg/Max metrics' default value2.2.031KIP-382: MirrorMaker 2.02.4.0 (WIP)32KIP-380: Detect outdated control requests and bounced brokers using broker generation2.2.033KIP-377: TopicCommand to use AdminClient2.2.034KIP-376: Implement AutoClosable on appropriate classes that want to be used in a try-with-resource statement2.2.035KIP-374: Add '--help' option to all available Kafka CLI commands2.2.036KIP-372: Naming Repartition Topics for Joins and Grouping2.1.037KIP-371: Add a configuration to build custom SSL principal name2.2.038KIP 368: Allow SASL Connections to Periodically Re-Authenticate2.2.039KIP-367 Introduce close(Duration) to Producer and AdminClient instead of close(long, TimeUnit)2.2.040KIP-366: Make FunctionConversions deprecated2.1.041KIP-365: Materialized, Serialized, Joined, Consumed and Produced with implicit Serde2.1.042KIP-361: Add Consumer Configuration to Disable Auto Topic Creation2.3.043KIP-359: Verify leader epoch in produce requests2.4.0 (WIP)44KIP-358: Migrate Streams API to Duration instead of long ms times2.1.045KIP-357: Add support to list ACLs per principal2.1.046KIP-356: Add withCachingDisabled() to StoreBuilder2.1.047KIP-354: Add a Maximum Log Compaction Lag2.3.048KIP-353: Improve Kafka Streams Timestamp Synchronization2.1.049KIP-351: Add --under-min-isr option to describe topics command2.3.050KIP-346: Improve LogCleaner behavior on error2.151KIP-345: Introduce static membership protocol to reduce consumer rebalances2.4.0 (WIP), partially available in 2.3.052KIP-342 Add support for custom SASL extensions in OAuthBearer authentication2.1.053KIP-341: Update Sticky Assignor's User Data Protocol2.3.054KIP-340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file2.1.055KIP-339: Create a new IncrementalAlterConfigs API2.3.0 56KIP-338 Support to exclude the internal topics in kafka-topics.sh command2.1.0 57KIP-336: Consolidate ExtendedSerializer/Serializer and ExtendedDeserializer/Deserializer2.1.058KIP-332: Update AclCommand to use AdminClient API2.1.059KIP-331 Add default implementation to close() and configure() for Serializer, Deserializer and Serde2.3.060KIP-330: Add retentionPeriod in SessionBytesStoreSupplier2.1.061KIP-328: Ability to suppress updates for KTables2.1.0 (partially implemented) / 2.3.0 (WIP)62KIP-324: Add method to get metrics() in AdminClient2.1.063KIP-322: Return new error code for DeleteTopics API when topic deletion disabled.2.1.064KIP-321: Update TopologyDescription to better represent Source and Sink Nodes2.1.065KIP-320: Allow fetchers to detect and handle log truncation2.1.0 (partially implemented) / 2.2.066KIP-319: Replace segments with segmentInterval in WindowBytesStoreSupplier2.1.067KIP-313: Add KStream.flatTransform and KStream.flatTransformValues 

2.2.0 (partially implemented)

/ 2.3.0

68KIP-312 Add Overloaded StreamsBuilder Build Method to Accept java.util.Properties2.1.069KIP-308: Support dynamic update of max.connections.per.ip/max.connections.per.ip.overrides2.1.070KIP-307: Allow to define custom processor names with KStreams DSL2.3.0 (partial)71KIP-306: Configuration for Delaying Response to Failed Authentication2.1.072KIP-305: Add Connect primitive number converters2.0.073KIP-303: Add Dynamic Routing in Streams Sink2.0.074KIP-302 - Enable Kafka clients to use all DNS resolved IP addresses2.1.075KIP-300: Add Windowed KTable API in StreamsBuilder2.4.076KIP-298: Error Handling in Connect2.0.077KIP-297: Externalizing Secrets for Connect Configurations2.0.078KIP-295 Add Streams Configuration Allowing for Optional Topology Optimization2.0.079KIP-294 - Enable TLS hostname verification by default2.0.080KIP-292: Add transformValues() method to KTable2.0.081KIP-290: Support for Prefixed ACLs2.0.082KIP-289: Improve the default group id behavior in KafkaConsumer2.2.083KIP-285: Connect Rest Extension Plugin2.0.084KIP-284: Set default retention ms for Streams repartition topics to Long.MAX_VALUE2.0.085KIP-283: Efficient Memory Usage for Down-Conversion2.0.086KIP-282: Add the listener name to the authentication context2.0.087KIP-281: ConsumerPerformance: Increase Polling Loop Timeout and Make It Reachable by the End User2.0.088KIP-279: Fix log divergence between leader and follower after fast leader fail over2.0.089KIP-278 - Add version option to Kafka's commands2.0.090KIP-277 - Fine Grained ACL for CreateTopics API2.0.0 91KIP-276 - Add StreamsConfig prefix for different consumers2.0.092KIP-274: Kafka Streams Skipped Records Metrics2.0.093KIP-272: Add API version tag to broker's RequestsPerSec metric2.0.094KIP-270 - A Scala Wrapper Library for Kafka Streams2.0.095KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade2.0.096KIP-267: Add Processor Unit Test Support to Kafka Streams Test Utils2.0.097KIP-266: Fix consumer indefinite blocking behavior2.0.098KIP-265: Make Windowed Serde to public APIs2.0.099KIP-261: Add Single Value Fetch in Window Stores2.0.0100KIP-258: Allow to Store Record Timestamps in RocksDB2.3.0 (partially implemented)101KIP-257 - Configurable Quota Management2.0.0102KIP-255: OAuth Authentication via SASL/OAUTHBEARER2.0.0103KIP-251: Allow timestamp manipulation in Processor API2.0.0104KIP-249: Add Delegation Token Operations to KafkaAdminClient2.0.0105KIP-247: Add public test utils for Kafka Streams1.1.0106KIP-245: Use Properties instead of StreamsConfig in KafkaStreams constructor2.0.0107KIP-244: Add Record Header support to Kafka Streams Processor API2.0.0108KIP-243: Make ProducerConfig and ConsumerConfig constructors public1.1.0109KIP-239 Add queryableStoreName() to GlobalKTable1.1.0110KIP-238: Expose Kafka cluster ID in Connect REST API1.1.0111KIP-237: More Controller Health Metrics2.0.0112KIP-235: Add DNS alias support for secured connection2.1.0113KIP-233: Simplify StreamsBuilder#addGlobalStore1.1.0114KIP-231: Improve the Required ACL of ListGroups API2.1.0115KIP-229: DeleteGroups API1.1.0116KIP-227 - Introduce Incremental FetchRequests to Increase Partition Scalability1.1.0117KIP-226 - Dynamic Broker Configuration1.1.0118KIP-225 - Use tags for consumer “records.lag” metrics1.1.0119KIP-224: Add configuration parameter `retries` to Streams API1.1.0120KIP-223 - Add per-topic min lead and per-partition lead metrics to KafkaConsumer2.0.0121KIP-222 - Add Consumer Group operations to Admin API2.0.0122KIP-220: Add AdminClient into Kafka Streams' ClientSupplier1.1.0123KIP-219 - Improve quota communication2.0.0124KIP-218: Make KafkaFuture.Function java 8 lambda compatible1.1.0125KIP-215: Add topic regex support for Connect sinks1.1.0126KIP-214: Add zookeeper.max.in.flight.requests config to the broker1.1.0127KIP-213 Support non-key joining in KTable2.4.0 (WIP)128KIP-212: Enforce set of legal characters for connector names1.1.0129KIP-211: Revise Expiration Semantics of Consumer Group Offsets2.1.0130KIP-210 - Provide for custom error handling when Kafka Streams fails to produce1.1.0131KIP-208: Add SSL support to Kafka Connect REST interface1.1.0132KIP-207: Offsets returned by ListOffsetsResponse should be monotonically increasing even during a partition leader change2.2.0133KIP-206: Add support for UUID serialization and deserialization2.1.0134KIP-205: Add all() and range() API to ReadOnlyWindowStore1.1.0135KIP-204 : Adding records deletion operation to the new Admin Client API1.1.0136KIP-203: Add toLowerCase support to sasl.kerberos.principal.to.local rule 1.1.0137KIP-202: Move merge() from StreamsBuilder to KStream1.0.0138KIP-198: Remove ZK dependency from Streams Reset Tool1.0.0139KIP-197 Connect REST API should include the connector type when describing a connector1.0.0140KIP-196: Add metrics to Kafka Connect framework1.0.0141KIP-195: AdminClient.createPartitions1.0.0142KIP-192 : Provide cleaner semantics when idempotence is enabled1.0.0143KIP-191: KafkaConsumer.subscribe() overload that takes just Pattern1.0.0144KIP-190: Handle client-ids consistently between clients and brokers1.0.0145KIP-189: Improve principal builder interface and add support for SASL1.0.0146KIP-188 - Add new metrics to support health checks1.0.0147KIP-187 - Add cumulative count metric for all Kafka rate metrics1.0.0148KIP-186: Increase offsets retention default to 7 days2.0.0149KIP-183 - Change PreferredReplicaLeaderElectionCommand to use AdminClient2.2.0150KIP-182: Reduce Streams DSL overloads and allow easier use of custom storage engines1.0.0151KIP-180: Add a broker metric specifying the number of consumer group rebalances in progress1.1.0152KIP-177: Consumer perf tool should count rebalance time1.0.0153KIP-176: Remove deprecated new-consumer option for tools2.0.0154KIP-175: Additional '--describe' views for ConsumerGroupCommand1.1.0155KIP-174 - Deprecate and remove internal converter configs in WorkerConfig2.0.0156KIP-173: Add prefix to StreamsConfig to enable setting default internal topic configs1.0.0157KIP-171 - Extend Consumer Group Reset Offset for Stream Application1.1.0158KIP-168: Add GlobalTopicCount and GlobalPartitionCount metric per cluster1.0.0159KIP-167: Add interface for the state store restoration process1.0.0160

KIP-164 - Add UnderMinIsrPartitionCount and per-partition UnderMinIsr metrics

1.0.0161

KIP-163: Lower the Minimum Required ACL Permission of OffsetFetch

1.0.0162KIP-162: Enable topic deletion by default1.0.0163KIP-161: streams deserialization exception handlers1.0.0164KIP-160: Augment KStream.print(), KStream.writeAsText() to allow users pass in extra parameters in the printed string1.0.0165KIP-157 - Add consumer config options to streams reset tool1.0.0166KIP-156 Add option "dry run" to Streams application reset tool0.11.0.0167KIP-155 - Add range scan for windowed state stores0.11.0.0168KIP-154 Add Kafka Connect configuration properties for creating internal topics0.11.0.0169KIP-153: Include only client traffic in BytesOutPerSec metric0.11.0.0170KIP-152 - Improve diagnostics for SASL authentication failures1.0.0171KIP-151 Expose Connector type in REST API0.11.0.0172KIP-150 - Kafka-Streams Cogroup2.4.0 (WIP)173KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner1.1.0 (partially implemented)
(WIP for 2.4.0)174KIP-146 - Classloading Isolation in Connect

0.11.0.0

175KIP-145 - Expose Record Headers in Kafka Connect1.1.0176KIP-144: Exponential backoff for broker reconnect attempts0.11.0.0177KIP-143: Controller Health Metrics0.11.0.0178KIP-140: Add administrative RPCs for adding, deleting, and listing ACLs0.11.0.0179KIP-138: Change punctuate semantics1.0.0180KIP-137: Enhance TopicCommand --describe to show topics marked for deletion0.11.0.0181KIP-136: Add Listener name to SelectorMetrics tags0.11.0.0182KIP-134: Delay initial consumer group rebalance0.11.0.0183KIP-133: Describe and Alter Configs Admin APIs0.11.0.0184KIP-130: Expose states of active tasks to KafkaStreams public API1.0.0185KIP-129: Streams Exactly-Once Semantics0.11.0.0186KIP-128: Add ByteArrayConverter for Kafka Connect0.11.0.0187KIP-126 - Allow KafkaProducer to split and resend oversized batches.0.11.0.0188KIP-124 - Request rate quotas0.11.0.0189KIP-123: Allow per stream/table timestamp extractor0.11.0.0190KIP-122: Add Reset Consumer Group Offsets tooling0.11.0.0191KIP-121: Add KStream peek method

0.11.0.0

192KIP-120: Cleanup Kafka Streams builder API1.0.0193KIP-119: Drop Support for Scala 2.10 in Kafka 0.110.11.0.0194KIP-118: Drop Support for Java 72.0.0195KIP-117: Add a public AdminClient API for Kafka admin operations0.11.0.0196KIP-115: Enforce offsets.topic.replication.factor upon __consumer_offsets auto topic creation0.11.0.0197KIP-114: KTable state stores and improved semantics0.11.0.0198KIP-113: Support replicas movement between log directories1.1.0199KIP-112: Handle disk failure for JBOD1.0.0200KIP-110: Add Codec for ZStandard Compression2.1.0201KIP-109: Old Consumer Deprecation0.11.0.0202KIP-108: Create Topic Policy0.10.2.0203KIP-107: Add deleteRecordsBefore() API in AdminClient0.11.0.0204

KIP-106 - Change Default unclean.leader.election.enabled from True to False

0.11.0.0205KIP-105: Addition of Record Level for Sensors0.10.2.0206KIP-104: Granular Sensors for Streams 0.10.2.0207KIP-103: Separation of Internal and External traffic0.10.2.0208KIP-102 - Add close with timeout for consumers0.10.2.0209KIP-101 - Alter Replication Protocol to use Leader Epoch rather than High Watermark for Truncation0.11.0.0210KIP-100 - Relax Type constraints in Kafka Streams API0.10.2.0211KIP-99: Add Global Tables to Kafka Streams0.10.2.0212KIP-98 - Exactly Once Delivery and Transactional Messaging0.11.0.0213KIP-97: Improved Kafka Client RPC Compatibility Policy0.10.2.0214KIP-96 - Add per partition metrics for in-sync and assigned replica count0.10.2.0215KIP-94 Session Windows0.10.2.0216KIP-93: Improve invalid timestamp handling in Kafka Streams0.10.2.0217KIP-92 - Add per partition lag metrics to KafkaConsumer0.10.2.0218KIP-91 Provide Intuitive User Timeouts in The Producer2.1.0219KIP-90 - Remove zkClient dependency from Streams0.10.2.0220KIP-89: Allow sink connectors to decouple flush and offset commit0.10.2.0221KIP-88: OffsetFetch Protocol Update0.10.2.0222KIP-86: Configurable SASL callback handlers2.0.0223KIP-85: Dynamic JAAS configuration for Kafka clients0.10.2.0224KIP-84: Support SASL SCRAM mechanisms0.10.2.0225KIP-82 - Add Record Headers0.11.0.0226KIP-81: Bound Fetch memory usage in the consumer2.4.0 (WIP)227KIP-79 - ListOffsetRequest/ListOffsetResponse v1 and add timestamp search methods to the new consumer0.10.1.0228KIP-78: Cluster Id0.10.1.0229KIP-77: Improve Kafka Streams Join Semantics0.10.2.0230KIP-75 - Add per-connector Converters0.10.1.0231KIP-74: Add Fetch Response Size Limit in Bytes0.10.1.0232KIP-73: Replication Quotas0.10.1.0233KIP-72: Allow putting a bound on memory consumed by Incoming request 1.0.0234KIP-71: Enable log compaction and deletion to co-exist0.10.1.0235KIP-70: Revise Partition Assignment Semantics on New Consumer's Subscription Change0.10.1.0236KIP-67: Queryable state for Kafka Streams0.10.1.0237KIP-66: Single Message Transforms for Kafka Connect0.10.2.0 / 0.11.0.0238KIP-65: Expose timestamps to Connect0.10.1.0239KIP-63: Unify store and downstream caching in streams0.10.1.0240KIP-62: Allow consumer to send heartbeats from a background thread0.10.1.0241KIP-60 - Make Java client classloading more flexible0.10.1.0242KIP-58 - Make Log Compaction Point Configurable0.10.1.0243KIP-57 - Interoperable LZ4 Framing0.10.0.0244KIP-56: Allow cross origin HTTP requests on all HTTP methods0.10.0.0245KIP-55: Secure Quotas for Authenticated Users0.10.1.0246KIP-54: Sticky Partition Assignment Strategy0.11.0.0247KIP-52: Connector Control APIs0.10.0.0248KIP-51 - List Connectors REST API0.10.0.0249KIP-50 - Move Authorizer to o.a.k.common package0.10.1.0250KIP-48 Delegation token support for Kafka1.1.0251KIP-45 - Standardize all client sequence interaction on j.u.Collection.0.10.0.0252KIP-43: Kafka SASL enhancements0.10.0.0253KIP-42: Add Producer and Consumer Interceptors0.10.0.0254KIP-41: Consumer Max Records0.10.0.0255KIP-40: ListGroups and DescribeGroup0.9.0.0256KIP-38: ZooKeeper Authentication0.9.0.0257KIP-36 - Rack aware replica assignment0.10.0.0258KIP-35 - Retrieving protocol version0.10.0.0259KIP-33 - Add a time based log index0.10.1.0260KIP-32 - Add timestamps to Kafka message0.10.0.0261KIP-31 - Move to relative offsets in compressed message sets0.10.0.0262KIP-28 - Add a processor client0.10.0.0263KIP-26 - Add Kafka Connect framework for data import/export0.9.0.0264KIP-25 - System test improvements0.9.0.0265KIP-22 - Expose a Partitioner interface in the new producer0.9.0.0266KIP-21 - Dynamic Configuration0.9.0.0 (WIP)267KIP-20 Enable log preallocate to improve consume performance under windows and some old Linux file system0.9.0.0268KIP-19 - Add a request timeout to NetworkClient0.9.0.0269KIP-16 - Automated Replica Lag Tuning0.9.0.0270KIP-15 - Add a close method with a timeout in the producer0.9.0.0271KIP-13 - Quota Design0.9.0.0272KIP-12 - Kafka Sasl/Kerberos and SSL implementation0.9.0.0273KIP-11 - Kafka Authorizer design0.9.0.0274KIP-8 - Add a flush method to the producer API0.9.0.0275KIP-4 - Metadata Protocol Changes0.10.0.0276KIP-4 - Command line and centralized administrative operations0.9.0.0, 0.10.0.0, 0.10.1.0277KIP-3 - Mirror Maker Enhancement0.9.0.0278KIP-2 - Refactor brokers to allow listening on multiple ports and IPs0.9.0.0279KIP-1 - Remove support of request.required.acks0.9.0.0

...

1019: Expose method to determine Metric Measurability

Accepted, Targeting 3.8.0

KIP-1013: Drop broker and tools support for Java 11 in Kafka 4.0 (deprecate in 3.7)

4.0.0

KIP-1012: The need for a Kafka 3.8.x release

3.8.0

KIP-1007: Introduce Remote Storage Not Ready Exception

3.8.0

KIP-1005: Expose EarliestLocalOffset and TieredOffset

3.8.0

KIP-1004: Enforce tasks.max property in Kafka Connect

3.8.0

KIP-1001: Add CurrentControllerId Metric 

3.7.0

KIP-1000: List Client Metrics Configuration Resources

3.7.0

KIP-996: Pre-Vote

TBD (targeting 3.8.0/4.0.0)

KIP-994: Minor Enhancements to ListTransactions and DescribeTransactions APIs

TBD (targeting 3.8.0)

KIP-993: Allow restricting files accessed by File and Directory ConfigProviders

3.8.0
KIP-992: Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery

3.7.0

KIP-988: Streams Standby Task Update Listener

3.7.0

KIP-985: Add reverseRange and reverseAll query over kv-store in IQv2

3.7.0

KIP-980: Allow creating connectors in a stopped state

3.7.0

KIP-979: Allow independently stop KRaft processes

TBD (targeting 3.7.0)

KIP-978: Allow dynamic reloading of certificates with different DN / SANs

3.7.0

KIP-977: Partition-level Throughput Metrics

TBD (targeting 3.8.0)

KIP-976: Cluster-wide dynamic log adjustment for Kafka Connect

3.7.0

KIP-975: Docker Image for Apache Kafka

3.7.0

KIP-974: Docker Image for GraalVM based Native Kafka Broker

TBD (targeting 3.7.0)

KIP-970: Deprecate and remove Connect's redundant task configurations endpoint

3.7.0 (deprecation)

4.0.0 (removal)

KIP-968: Support single-key_multi-timestamp Interactive Queries (IQv2) for Versioned State Stores 

3.7.0

KIP-966: Eligible Leader Replicas

TBD

KIP-963: Additional metrics in Tiered Storage

3.7.0

KIP-962: Relax non-null key requirement in Kafka Streams

3.7.0

KIP-960: Support single-key_single-timestamp Interactive Queries (IQv2) for Versioned State Stores

3.7.0

KIP-959: Add BooleanConverter to Kafka Connect

3.7.0

KIP-956 Tiered Storage Quotas

Accepted, Targeting 3.8.0

KIP-954: expand default DSL store configuration to custom types3.7.0

KIP-951: Leader discovery optimisations for the client

3.7.0

KIP-949: Add flag to enable the usage of topic separator in MM2 DefaultReplicationPolicy

Adopted, targeting 3.6

KIP-942: Add Power(ppc64le) support

TBD

KIP-941: Range queries to accept null lower and upper bounds3.6.0
KIP-938: Add more metrics for measuring KRaft performanceTBD
KIP-937: Improve Message Timestamp Validation3.6.0
KIP-927: Improve the kafka-metadata-quorum output3.6.0
KIP-925: Rack aware task assignment in Kafka Streams3.7.0 (completed)
3.6.0 (partially implemented)
KIP-923: Add A Grace Period to Stream Table Join3.6.0

KIP-919: Allow AdminClient to Talk Directly with the KRaft Controller Quorum and add Controller Registration

3.7.0
KIP-917: Additional custom metadata for remote log segment3.6.0
KIP-915: Txn and Group Coordinator Downgrade Foundation3.5.0
KIP-914: DSL Processor Semantics for Versioned Stores3.5.0
KIP-911: Add source tag to Mirror source metric3.5.0
KIP-909: DNS Resolution Failure Should Not Fail the ClientsTBD
KIP-907: Add Boolean Serde to public interface3.5.0
KIP-906: Tools migration guidelines3.5.0
KIP-904: Kafka Streams - Guarantee subtractor is called before adder if key has not changed3.5.0
KIP-903: Replicas with stale broker epoch should not be allowed to join the ISR3.5.0
KIP-902: Upgrade Zookeeper to 3.8.23.6.0
KIP-900: KRaft kafka-storage.sh API additions to support SCRAM for Kafka Brokers3.5.0
KIP-898: Modernize Connect plugin discovery3.6.0
KIP-896: Remove old client protocol API versions in Kafka 4.0

3.7.0 - new metric & request log attribute

4.0.0 - actual removal

KIP-894: Use incrementalAlterConfig for syncing topic configurations 3.5.0
KIP-893: The Kafka protocol should support nullable structs3.5.0
KIP-892: Transactional Semantics for StateStores

TBD (targeting 3.7.0)

KIP-890: Transactions Server-Side DefenseTDB
KIP-889: Versioned State Stores3.5.0
KIP-887: Add ConfigProvider to make use of environment variables3.5.0
KIP-884: Add config to configure KafkaClientSupplier in Kafka Streams3.5.0
KIP-881: Rack-aware Partition Assignment for Kafka Consumers

3.4.0 (protocol)

3.5.0 (assignors)

KIP-878: Autoscaling for Stateless & Statically Partitioned StreamsTBD
KIP-876: Time based cluster metadata snapshots3.4.0
KIP-875: First-class offsets support in Kafka Connect

3.5.0 (STOPPED state, offsets read API)

3.6.0 (offsets alter/reset APIs)

KIP-872: Add Serializer#serializeToByteBuffer() to reduce memory copyingAccepted
KIP-869: Improve Streams State Restoration Visibility3.5.0 (partially implemented) / TDB
KIP-868 Metadata Transactions3.6.0
KIP-866 ZooKeeper to KRaft Migration3.4.0
KIP-865: Support --bootstrap-server in kafka-streams-application-reset3.4.0
KIP-863: Reduce CompletedFetch#parseRecord() memory copyTBD
KIP-862: Self-join optimization for stream-stream joins3.4.0
KIP-859: Add Metadata Log Processing Error Related Metrics3.3.0
KIP-858: Handle JBOD broker disk failure in KRaft3.7.0
KIP-854 Separate configuration for producer ID expiry3.4.0
KIP-852: Optimize calculation of size for log in remote tierAccepted. Targeting 3.6.0.
KIP-851: Add requireStable flag into ListConsumerGroupOffsetsOptions3.3.0
KIP-848: The Next Generation of the Consumer Rebalance ProtocolTBD
KIP-847: Add ProducerIdCount metrics3.5.0
KIP-846: Source/sink node metrics for Consumed/Produced throughput in Streams3.3.0
KIP-843: Adding addMetricIfAbsent method to Metrics3.3.0
KIP-841: Fenced replicas should not be allowed to join the ISR in KRaft3.3.0
KIP-840: Config file option for MessageReader/MessageFormatter in ConsoleProducer/ConsoleConsumer3.4.0
KIP-837: Allow MultiCasting a Result Record.3.4.0
KIP-836: Expose replication information of the cluster metadata3.3.0
KIP-835: Monitor KRaft Controller Quorum Health3.3.0
KIP-834: Pause / Resume KafkaStreams Topologies3.3.0
KIP-833: Mark KRaft as Production Ready3.3.0
KIP-831: Add metric for log recovery progress3.3.0
KIP-830: Allow disabling JMX Reporter3.4.0
KIP-827: Expose logdirs total and usable space via Kafka API3.3.0
KIP-825: introduce a new API to control when aggregated results are produced3.3.0
KIP-824: Allowing dumping segmentlogs limiting the batches in the output3.3.0
KIP-821: Connect Transforms support for nested structuresTBD
KIP-820: Extend KStream process with new Processor API3.3.0
KIP-815: Support max-timestamp in GetOffsetShell3.2.0
KIP-814: Static membership protocol should let the leader skip assignment3.2.0
KIP-813: Shareable State StoresTBD
KIP-812: Introduce another form of the `KafkaStreams.close()` API that forces the member to leave the consumer group3.3.0
KIP-811: Add config repartition.purge.interval.ms to Kafka Streams3.2.0

KIP-810: Allow producing records with null values in Kafka Console Producer

3.2.0
KIP-808: Add support for different unix precisions in TimestampConverter SMT3.2.0
KIP-806: Add session and window query over kv-store in IQv23.2.0
KIP-805: Add range and scan query over kv-store in IQv23.2.0
KIP-801: Implement an Authorizer that stores metadata in __cluster_metadata3.2.0
KIP-800: Add reason to JoinGroupRequest and LeaveGroupRequest3.2.0
KIP-798: Add possibility to write kafka headers in Kafka Console Producer3.2.0
KIP-797: Accept duplicate listener on port for IPv4/IPv63.6.0
KIP-796: Interactive Query v2TBD
KIP-794: Strictly Uniform Sticky Partitioner3.3.0
KIP-793: Allow sink connectors to be used with topic-mutating SMTs3.6.0
KIP-792: Add "generation" field into consumer protocol3.4.0
KIP-791: Add Record Metadata to StateStoreContext3.2.0
KIP-788: Allow configuring num.network.threads per listener3.2.0
KIP-787: MM2 manage Kafka resources with custom Admin implementation.3.4.0
KIP-784: Add top-level error code field to DescribeLogDirsResponse3.2.0
KIP-783: Add TaskId field to StreamsException3.1.0
KIP-779: Allow Source Tasks to Handle Producer Exceptions3.2.0
KIP-778: KRaft to KRaft Upgrades3.3.0
KIP-775: Custom partitioners in foreign key joins3.1.0
KIP-773: Differentiate consistently metric latency measured in millis and nanos3.1.0
KIP 771: KRaft brokers without the "controller" role should not expose controller metrics3.0.0
KIP-770: Replace "buffered.records.per.partition" with "input.buffer.max.bytes"3.4.0 (WIP)

KIP-769: Connect APIs to list all connector plugins and retrieve their configuration definitions

3.2.0
KIP-768: Extend SASL/OAUTHBEARER with Support for OIDC3.1.0
KIP-766: fetch/findSessions queries with open endpoints for SessionStore/WindowStore3.1.0
KIP-764: Configurable backlog size for creating Acceptor3.2.0
KIP-763: Range Queries with Open Endpoints3.1.0
KIP-761: Add Total Blocked Time Metric to Streams3.1.0
KIP-751: Drop support for Scala 2.12 in Kafka 4.0 (deprecate in 3.0)3.0.0 (deprecation notice), 4.0.0 (support withdrawal)
KIP-750: Drop support for Java 8 in Kafka 4.0 (deprecate in 3.0)3.0.0 (deprecation notice), 4.0.0 (support withdrawal)
KIP-748: Add Broker Count Metrics3.1.0
KIP-746: Revise KRaft Metadata Records3.0.0
KIP-745: Connect API to restart connector and tasks3.0.0
KIP-744: Migrate TaskMetadata and ThreadMetadata to an interface with internal implementation3.0.0
KIP-743: Remove config value 0.10.0-2.4 of Streams built-in metrics version config3.0.0
KIP-741: Change default serde to be null3.0.0
KIP-740: Clean up public API in TaskId3.0.0
KIP-738: Removal of Connect's internal converter properties3.0.0
KIP-735: Increase default consumer session timeout3.0.0
KIP-734: Improve AdminClient.listOffsets to return timestamp and offset for the record with the largest timestamp3.0.0
KIP-733: change Kafka Streams default replication factor config3.0.0
KIP-732: Deprecate eos-alpha and replace eos-beta with eos-v23.0.0
KIP-730: Producer ID generation in KRaft mode3.0.0
KIP-726: Make the "cooperative-sticky, range" as the default assignorWIP
KIP-725: Streamlining configurations for WindowedSerializer and WindowedDeserializer.3.0.0
KIP-724: Drop support for message formats v0 and v1

3.0.0 (deprecation)

4.0.0 (removal)

KIP-722: Enable connector client overrides by default3.0.0
KIP-721: Enable connector log contexts in Connect Log4j configuration3.0.0
KIP-720: Deprecate MirrorMaker v13.0.0
KIP-719: Deprecate Log4J AppenderWIP
KIP-716: Allow configuring the location of the offset-syncs topic with MirrorMaker23.0.0
KIP-715: Expose Committed offset in streams3.0.0
KIP-714: Client metrics and observability3.7.0
KIP-710: Full support for distributed mode in dedicated MirrorMaker 2.0 clusters3.5.0
KIP-709: Extend OffsetFetch requests to accept multiple group ids.3.3.0
KIP-708: Rack aware StandbyTask assignment for Kafka Streams3.2.0
KIP-707: The future of KafkaFuture3.0.0

KIP-704: Send a hint to the partition leader to recover the partition

3.2.0
KIP-700: Add Describe Cluster API2.8.0

KIP-699: Update FindCoordinator to resolve multiple Coordinators at a time

3.0.0
KIP-698: Add Explicit User Initialization of Broker-side State to Kafka StreamsWIP / inactive
KIP-696: Update Streams FSM to clarify ERROR state meaning2.8.0
KIP-695: Further Improve Kafka Streams Timestamp Synchronization3.0.0
KIP-691: Enhance Transactional Producer Exception HandlingWIP / inactive
KIP-690: Add additional configuration to control MirrorMaker 2 internal topics naming convention3.1.0
KIP-679: Producer will enable the strongest delivery guarantee by default3.0.0
KIP-689: Extend `StreamJoined` to allow more store configs2.8.0
KIP-684 - Support mutual TLS authentication on SASL_SSL listeners2.8.0
KIP-680: TopologyTestDriver should not require a Properties argument2.8.0
KIP-676: Respect logging hierarchy2.8.0
KIP-673: Emit JSONs with new auto-generated schema2.8.0
KIP-671: Introduce Kafka Streams Specific Uncaught Exception Handler2.8.0
KIP-666: Add Instant-based methods to ReadOnlySessionStore3.0.0
KIP-663: API to Start and Shut Down Stream Threads2.8.0
KIP-662: Throw Exception when Source Topics of a Streams App are Deleted2.7.0

KIP-661: Expose task configurations in Connect REST API

2.8.0
KIP-659: Improve TimeWindowedDeserializer and TimeWindowedSerde to handle window size2.8.0
KIP-654: Aborted transaction with non-flushed data should throw a non-fatal exception2.7.0
KIP-653: Upgrade log4j to log4j23.2.0 (WIP)
KIP-651 - Support PEM format for SSL certificates and private key2.7.0
KIP-648 Renaming getter method for Interactive Queries2.7.0
KIP-641 An new java interface to replace 'kafka.common.MessageReader'3.5.0
KIP-635: GetOffsetShell: support for multiple topics and consumer configuration override3.0
KIP-633: Deprecate 24-hour Default Grace Period for Windowed Operations in Streams3.0.0
KIP-632: Add DirectoryConfigProvider2.7.0
KIP-631: The Quorum-based Kafka Controller2.8.0
KIP-630: Kafka Raft Snapshot3.0.0
KIP-629: Use racially neutral terms in our codebase3.0.0

KIP-627: Expose Trogdor-specific JMX Metrics for Tasks and Agents

WIP
KIP-626: Rename StreamsConfig config variable name2.7.0
KIP-623: Add "internal-topics" option to streams application reset tool3.0.0
KIP-622: Add currentSystemTimeMs and currentStreamTimeMs to ProcessorContext3.0.0
KIP-621: Deprecate and replace DescribeLogDirsResult.all() and .values()2.7.0
KIP-620 Deprecate ConsumerConfig#addDeserializerToConfig and ProducerConfig#addSerializerToConfig2.7.0
KIP-618: Exactly-Once Support for Source Connectors

3.3.0 (almost everything)

3.2.0 (new admin client API)

KIP-617: Allow Kafka Streams State Stores to be iterated backwards2.7.0
KIP-616: Rename implicit Serdes instances in kafka-streams-scala2.7.0
KIP-614: Add Prefix Scan support for State Stores2.8.0
KIP-613: Add end-to-end latency metrics to Streams2.6.0 (INFO node-level metrics), remainder 2.7.0 (TRACE store-level metrics) 
KIP-612: Ability to Limit Connection Creation Rate on Brokers2.7.0 / 2.8.0
KIP-610: Error Reporting in Sink Connectors2.6.0
KIP-608 - Expose Kafka Metrics in AuthorizerWIP
KIP-607: Add Metrics to Kafka Streams to Report Properties of RocksDB2.7.0
KIP-606: Add Metadata Context to MetricsReporter2.6.0
KIP-605: Expand Connect Worker Internal Topic Settings2.6.0
KIP-604: Remove ZooKeeper Flags from the Administrative Toolsone part in 2.6.0, most parts scheduled for 3.0
KIP-602: Change default value for client.dns.lookup2.6.0
KIP-601: Configurable socket connection timeout2.7.0
KIP-599: Throttle Create Topic, Create Partition and Delete Topic Operations2.7.0
KIP-597: MirrorMaker2 internal topics Formatters2.7.0
KIP-594: Expose output topic names from TopologyTestDriver2.6.0
KIP-591: Add Kafka Streams config to set default state store3.2.0 (WIP)
KIP-590: Redirect Zookeeper Mutation Protocols to The Controller2.8.0 (WIP)
KIP-589 Add API to update Replica state in Controller2.8.0 (WIP)
KIP-588: Allow producers to recover gracefully from transaction timeouts2.8.0 (WIP)
KIP-586: Deprecate commit records without record metadata2.6.0
KIP-585: Filter and Conditional SMTs2.6.0
KIP-584: Versioning scheme for features2.7.0
KIP-581: Value of optional null field which has default value3.5.0
KIP-580: Exponential Backoff for Kafka Clients3.7.0
KIP-577: Allow HTTP Response Headers to be Configured for Kafka Connect2.6.0
KIP-574: CLI Dynamic Configuration with file input2.6.0
KIP-573: Enable TLSv1.3 by default2.6.0
KIP-572: Improve timeouts and retries in Kafka Streams2.8.0 (partially implemented in 2.7.0)
KIP-571: Add option to force remove members in StreamsResetter2.6.0

KIP-570: Add leader epoch in StopReplicaRequest

2.6.0
KIP-569: DescribeConfigsResponse - Update the schema to include additional metadata information of the field2.6.0
KIP-568: Explicit rebalance triggering on the Consumer2.6.0
KIP-562: Allow fetching a key from a single partition rather than iterating over all the stores on an instance2.5.0 
KIP-559: Make the Kafka Protocol Friendlier with L7 Proxies2.5.0
KIP-558: Track the set of actively used topics by connectors in Kafka Connect2.5.0
KIP-557: Add emit on change support for Kafka StreamsImplemented for KTable source nodes in 2.6.0, full implementation planned for subsequent releases.
KIP-555: Deprecate direct Zookeeper access in Kafka administrative tools2.5.0
KIP-554: Add Broker-side SCRAM Config API2.7.0
KIP-553: Disable all SSL protocols except TLSV1.2 by default.2.5.0
KIP-551: Expose disk read and write metrics2.6.0
KIP-546: Add Client Quota APIs to the Admin Client2.6.0 (describe/alter), remainder pending (resolve)
KIP-545: support automated consumer offset sync across clusters in MM 2.02.7.0
KIP-544: Make metrics exposed via JMX configurable2.5.0
KIP-543: Expand ConfigCommand's non-ZK functionality2.5.0
KIP-541: Create a fetch.max.bytes configuration for the broker2.5.0
KIP-538: Add a metric tracking the number of open connections with a given SSL cipher type2.5.0
KIP-537: Increase default zookeeper session timeout2.5.0
KIP-535: Allow state stores to serve stale reads during rebalance2.5.0
KIP-534: Reorganize checkpoint system in log cleaner to per partition 3.1.0
KIP-533: Add default api timeout to AdminClient2.5.0

KIP-532: Broker Consumer Lag metrics in size and time

WIP / inactive
KIP-531: Drop support for Scala 2.11 in Kafka 2.5Initial version in 2.5.0, remainder in later release (WIP)
KIP-530: Consider renaming 'UsePreviousTimeOnInvalidTimeStamp' class to 'UsePartitionTimeOnInvalidTimeStamp'2.5.0
KIP-528: Deprecate PartitionGrouper configuration and interface2.4.0
KIP-527: Add VoidSerde to Serdes2.5.0
KIP-526: Reduce Producer Metadata Lookups for Large Number of Topics2.5.0
KIP-525 - Return topic metadata and configs in CreateTopics response2.4.0 
KIP-524: Allow users to choose config source when describing configs2.5.0
KIP-523: Add KStream#toTable to the Streams DSL2.5.0
KIP-521: Enable redirection of Connect's log4j messages to a file by default2.4.0
KIP-519: Make SSL context/engine configuration extensible2.6.0
KIP-518: Allow listing consumer groups per state2.6.0
KIP-517: Add consumer metrics to observe user poll behavior2.4.0
KIP-516: Topic Identifiers & Topic Deletion State ImprovementsSome in 2.8.0, remainder WIP
KIP-515: Enable ZK client to use the new TLS supported authentication2.5.0
KIP-514:Add a bounded flush() API to Kafka Producer2.5.0
KIP-511: Collect and Expose Client's Name and Version in the Brokers2.4.0 (protocol) / 2.5.0 (metric)
KIP-507: Securing Internal Connect REST Endpoints2.4.0
KIP-504 - Add new Java Authorizer Interface2.4.0 
KIP-503: Add metric for number of topics marked for deletion2.4.0 
KIP-501: Avoid out-of-sync or offline partitions when follower fetch requests not processed in timeWIP
KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum
KIP-499 - Unify connection name flag for command line tool2.5.0
KIP-497: Add inter-broker API to alter ISR2.7.0
KIP-496: Administrative API to delete consumer offsets2.4.0 
KIP-495: Dynamically Adjust Log Levels in Connect2.4.0
KIP-492: Add java security providers in Kafka Security config2.4.0 
KIP-488: Clean up Sum,Count,Total Metrics2.4.0
KIP-484: Expose metrics for group and transaction metadata loading duration2.4.0 
KIP-482: The Kafka Protocol should Support Optional Tagged Fields2.4.0 
KIP-481: SerDe Improvements for Connect Decimal type in JSON2.4.0
KIP-480: Sticky Partitioner2.4.0
KIP-479: Add StreamJoined config object to Join2.4.0
KIP-478 - Strongly typed Processor API2.7.0
KIP-476: Add Java AdminClient Interface2.4.0
KIP-475: New Metrics to Measure Number of Tasks on a Connector2.4.0
KIP-474: To deprecate WindowStore#put(key, value)2.4.0
KIP-471: Expose RocksDB Metrics in Kafka Streams

2.4.0 (metrics blocked due to RocksDB version added in 3.2.0)

KIP-470: TopologyTestDriver test input and output usability improvements2.4.0 
KIP-467: Augment ProduceResponse error messaging for specific culprit records2.4.0 (partially implemented) / 2.5.0
KIP-466: Add support for List<T> serialization and deserialization3.0.0
KIP-465: Add Consolidated Connector Endpoint to Connect REST API2.3.0
KIP-464: Defaults for AdminClient#createTopic2.4.0
KIP-462: Use local thread id for KStreams2.3.0
KIP-461: Improve Replica Fetcher behavior at handling partition failure2.3.0
KIP-460: Admin Leader Election RPC2.4.0
KIP-458: Connector Client Config Override Policy2.3.0
KIP-455: Create an Administrative API for Replica Reassignment2.4.0 (API/broker logic)
2.5.0 (tools changes)
KIP-454: Expansion of the ConnectClusterState interface2.3.0
KIP-453: Add close() method to RocksDBConfigSetter2.3.0
KIP-450: Sliding Window Aggregations in the DSL2.7.0
KIP-449: Add connector contexts to log messages in Connect workers 2.3.0
KIP-447: Producer scalability for exactly once semantics2.5.0  (broker, consumer, producer changes implemented) / 2.6.0
KIP-446: Add changelog topic configuration to KTable suppress2.6.0
KIP-445: In-memory Session Store2.3.0
KIP-444: Augment metrics for Kafka Streams2.4.0 (partially implemented) / 2.5.0 (partially implemented) / 2.6.0

KIP-443: Return to default segment.ms and segment.index.bytes in Streams repartition topics

2.3.0
KIP-442: Return to default max poll interval in Streams2.3.0
KIP-441: Smooth Scaling Out for Kafka Streams2.6.0
KIP-440: Extend Connect Converter to support headers2.4.0
KIP-437: Custom replacement for MaskField SMT2.6.0
KIP-436: Add a metric indicating start time2.3.0
KIP-434: Add Replica Fetcher and Log Cleaner Count Metrics2.4.0
KIP-431: Support of printing additional ConsumerRecord fields in DefaultMessageFormatter2.7.0
KIP-430 - Return Authorized Operations in Describe Responses2.3.0
KIP-429: Kafka Consumer Incremental Rebalance Protocol2.4.0
KIP-428: Add in-memory window store2.3.0
KIP-427: Add AtMinIsr topic partition category (new metric & TopicCommand option)2.3.0
KIP-425: Add some Log4J Kafka Appender Properties for Producing to Secured Brokers2.3.0
KIP-421: Support resolving externalized secrets in AbstractConfig2.3.0
KIP-420: Add Single Value Fetch in Session Stores2.2.0
KIP-418: A method-chaining way to branch KStream2.8.0
KIP-417: Allow JmxTool to connect to a secured RMI port2.3.0
KIP-415: Incremental Cooperative Rebalancing in Kafka Connect2.3.0
KIP-414: Expose Embedded ClientIds in Kafka Streams2.2.0
KIP-412: Extend Admin API to support dynamic application log levels2.4.0 

KIP-411: Make default Kafka Connect worker task client IDs distinct

2.3.0
KIP-405: Kafka Tiered StorageAccepted
KIP-402: Improve fairness in SocketServer processors2.2.0 (partially implemented) / 2.3.0
KIP-401: TransformerSupplier/ProcessorSupplier StateStore connecting2.6.0
KIP-399: Extend ProductionExceptionHandler to cover serialization exceptions3.5.0
KIP-396: Add Commit/List Offsets Operations to AdminClient2.5.0
KIP-394: Require member.id for initial join group request2.2.0
KIP-393: Time windowed serde to properly deserialize changelog input topic2.2.0
KIP-392: Allow consumers to fetch from closest replica2.4.0
KIP-390: Support Compression LevelWIP
KIP-389: Introduce a configurable consumer group size limit2.2.0
KIP-386: Standardize on Min/Avg/Max metrics' default value2.2.0
KIP-382: MirrorMaker 2.02.4.0

KIP-379: Multiple Consumer Group Management

2.4.0 
KIP-380: Detect outdated control requests and bounced brokers using broker generation2.2.0
KIP-377: TopicCommand to use AdminClient2.2.0
KIP-376: Implement AutoClosable on appropriate classes that want to be used in a try-with-resource statement2.2.0
KIP-374: Add '--help' option to all available Kafka CLI commands2.2.0
KIP-373: Allow users to create delegation tokens for other users3.3.0
KIP-372: Naming Repartition Topics for Joins and Grouping2.1.0
KIP-371: Add a configuration to build custom SSL principal name2.2.0

Please insert new rows in sorted order (by KIP number).

KIPCommentKIP-59: Proposal for a kafka broker commandSent emails to Dev discussion group. Work tracked under KAFKA-3663.KIP-125: ZookeeperConsumerConnector to KafkaConsumer Migration and RollbackKIP-131 - Add access to OffsetStorageReader from SourceConnectorKIP-135 : Send of null key to a compacted topic should throw non-retriable error back to userKIP 141 - ProducerRecord & SourceRecord: Add timestamp constructorsKIP-142: Add ListTopicsRequest to efficiently list all the topics in a clusterKIP-148: Add a connect timeout for clientKIP-158: Kafka Connect should allow source connectors to set topic-specific settings for new topicsKIP-159: Introducing Rich functions to StreamsKIP-166 - Add a tool to make amounts of replicas and leaders on brokers balancedKIP-169 - Lag-Aware Partition Assignment StrategyKIP-178: Size-based log directory selection strategy

KIP-185: Make exactly once in order delivery the default producer setting

KIP-193: Add SchemaBuilder.from(Schema)KIP-199: Add Kafka Connect offset toolKIP-201: Rationalising Policy interfacesKIP-209: Connection String SupportKIP-216: IQ should throw different exceptions for different errorsKIP-217: Expose a timeout to allow an expired ZK session to be re-createdKIP-221: Enhance KStream with Connecting Topic Creation and Repartition HintDiscussionKIP-228 Negative record timestamp supportVoting in progressKIP-234: add support for getting topic defaults from AdminClientKIP-236: Interruptible Partition Reassignment DiscussionKIP-240: AdminClient.listReassignments() AdminClient.describeReassignments()KIP-242: Mask password in Kafka Connect Rest API responseKIP-250 Add Support for Quorum-based Producer AcknowledgmentKIP-252 - Extend ACLs to allow filtering based on ip ranges and subnetsKIP-253: Support in-order message delivery with partition expansion DiscussionKIP-254: JsonConverter Exception HandelingKIP-259: Improve Streams DSL Timestamp Propagation SemanticsKIP-260: add primary join operation for Stream-Stream join (WIP)DraftKIP-264: Add a consumer metric to record raw fetch sizeVoting in progressKIP-271: Add NetworkClient redirectorDiscussionKIP-273: Kafka to support using ETCD beside ZookeeperDiscussionKIP-275 - Indicate "isClosing" in the SinkTaskContextVoting in progressKIP-280: Enhanced log compactionDiscussion

KIP-291: Separating controller connections and requests from the data plane

AcceptedKIP-293 Add new metrics for consumer/replication fetch requestsVoting in progressKIP-296: Add connector level configurability for producer/consumer client configs DiscussionKIP-301: Schema Inferencing for JsonConverterDiscussionKIP-304: Connect runtime mode improvements for container platformsDiscussionKIP-314: KTable to GlobalKTable Bi-directional JoinDiscussionKIP-315: Stream Join Sticky AssignorDiscussionKIP-316: Command-line overrides for ConnectDistributed worker propertiesDiscussionKIP-317: Add end-to-end data encryption functionality to Apache KafkaDiscussionKIP-325: Extend Consumer Group Command to Show Beginning OffsetsVoting in Progress345KIP-326: Schedulable KTable as Graph sourceDiscussionKIP-333: Add faster mode of rebalancingDiscussionKIP-334 - Include partitions in exceptions raised during consumer record deserialization/validationKIP-335: Consider configurations for KafkaStreams

Discussion

KIP-347: Enable batching in FindCoordinatorRequestDiscussionKIP-348 Eliminate null from SourceTask#poll()KIP-350: Allow kafka-topics.sh to take brokerid as parameter to show partitions associated with itKIP-356: Add KafkaConsumer fetch-error-rate and fetch-error-total metricsDiscussionKIP-360: Improve handling of unknown producerDiscussionKIP-362: Support Dynamic Session WindowDiscussionKIP-363: Allow performance tools to print final results to output fileDiscussion
KIP-369: Alternative Partitioner to Support "Always Round-Robin" Selection
Accepted
2.4.0
KIP
-370: Remove Orphan PartitionsDiscussionKIP-373: Allow users to create delegation tokens for other usersDiscussionKIP-375: Kafka Clients - make Metadata#TOPIC_EXPIRY_MS configurableDiscussionKIP-378: Enable Dependency Injection for Kafka Streams handlersDiscussion

KIP-379: Multiple Consumer Group Management

AcceptedKIP-381: Connect: Tell about records that had their offsets flushed in callback

Voting in progress (restarted 18th January 2019, due to no votes in first attempt)

KIP-383:  Pluggable interface for SSL FactoryVoting in progressKIP-384: Add config for incompatible changes to persistent metadataDiscussionKIP-385: Avoid throwing away prefetched dataDiscussionKIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

Discussion

KIP-388: Add observer interface to record request and responseDiscussionKIP-389: Introduce a configurable consumer group size limitAcceptedKIP-390: Allow fine-grained configuration for compressionDiscussion, JIRA exists with pull-requestKIP-391: Allow Producing with Offsets for Cluster ReplicationDiscussion, JIRA exists with pull-requestKIP-392: Allow consumers to fetch from closest replicaDiscussionKIP-395: Encypt-then-MAC Delegation token metadata

KIP-396: Add Commit/List Offsets Operations to AdminClient

DiscussionKIP-397: Add methods to override fetch offsets based on timestampDiscussionKIP-398: Support reading trust store from classpathDiscussionKIP-399: Extend ProductionExceptionHandler to cover serialization exceptionsDiscussionKIP-400: Improve exit status in case of errors in ConsoleProducerDiscussionKIP-401: TransformerSupplier/ProcessorSupplier StateStore connectingVoting in progressKIP-403: Increase ProducerPerformance precision by using nanoTimeDraftKIP-405: Kafka Tiered Storage

Discussion

KIP-406: GlobalStreamThread should honor custom offset policy.DiscussionKIP-407: Kafka Connect support override worker kafka api configuration with connector configuration that post by rest apiKIP-408: Add asynchronous processing to Kafka StreamsDiscussion

KIP-409: Allow creating under-replicated topics and partitions

KIP-410: Add metric for request handler thread pool utilization by request type

DiscussionKIP-416: Notify SourceTask of ACK'd offsets, metadataDiscussionKIP-418: A method-chaining way to branch KStreamDiscussion, JIRA exists with pull-requestKIP-419: Safely notify Kafka Connect SourceTask is stoppedVoting in progress, JIRA exists with pull requestKIP-421: Support resolving externalized secrets in AbstractConfigAcceptedKIP-422: Add support for client quota configuration in the Kafka Admin ClientDiscussion, JIRA exists with pull-requestKIP-423: Add JoinReason to Consumer Join Group Protocol

Discussion

KIP-424: Allow suppression of intermediate events based on wall clock timeDiscussionKIP-426: Persist Broker Id to ZookeeperDiscussionKIP-431: Support of printing additional ConsumerRecord fields in DefaultMessageFormatterVoting in progress, JIRA exists with pull-requestKIP-434: Add Replica Fetcher and Log Cleaner Count MetricsDiscussionKIP-435: Incremental Partition ReassignmentDiscussionKIP-437: Custom replacement for MaskField SMTVoting in progress, JIRA exists with pull-requestKIP-438: Expose task, connector IDs in Connect APIDiscussionKIP-439: Cleanup built-in Store interfacesDiscussionKIP-440: Extend Connect Converter to support headersVoting in progressKIP-444: Augment metrics for Kafka StreamsDiscussionKIP-448: Add State Stores Unit Test Support to Kafka Streams Test UtilsDiscussionKIP-450: Sliding Window Aggregations in the DSLDiscussionKIP-452: Tool to view cluster statusDiscussion, JIRA existsKIP-455: Create an Administrative API for Replica ReassignmentDiscussionKIP-456: Helper classes to make it simpler to write test logic with TopologyTestDriver

Discussion

KIP-457: Add DISCONNECTED status to Kafka StreamsDiscussion KIP-459: Improve KafkaStreams#closeDiscussionKIP-463: Auto-configure non-default Serdes passed alongside the TopologyBuilderUnder discussionKIP-466: Add support for List<T> serialization and deserializationUnder DiscussionKIP-468: Avoid decompression of record when validate record at server in the scene of inPlaceAssignment .Under discussionKIP-470: TopologyTestDriver test input and output usability improvementsUnder discussionKIP-471: Expose RocksDB Metrics in Kafka Streams

Accepted

KIP-473: Enable KafkaLog4JAppender to use SASL Authentication Callback HandlersUnder Discussion KIP-474: To deprecate WindowStore#put(key, value)Under DiscussionKIP-475: New Metrics to Measure Number of Tasks on a ConnectorVotingKIP-476: Add Java AdminClient InterfaceUnder DiscussionKIP-477: Add PATCH method for connector config in Connect REST APIUnder DiscussionKIP-478 - Strongly typed Processor APIUnder DiscussionKIP-479: Add Materialized to JoinUnder DiscussionKIP-480: Sticky PartitionerUnder Discussion

Dormant/inactive KIPs

Please insert new rows in sorted order (by KIP number).

...

Functionality can be realized with existing API although at a lower level. 

368: Allow SASL Connections to Periodically Re-Authenticate2.2.0
KIP-367 Introduce close(Duration) to Producer and AdminClient instead of close(long, TimeUnit)2.2.0
KIP-366: Make FunctionConversions deprecated2.1.0
KIP-365: Materialized, Serialized, Joined, Consumed and Produced with implicit Serde2.1.0
KIP-361: Add Consumer Configuration to Disable Auto Topic Creation2.3.0
KIP-360: Improve reliability of idempotent/transactional producer2.4.0 (partially implemented) / 2.5.0
KIP-359: Verify leader epoch in produce requests2.8.0 (WIP)
KIP-358: Migrate Streams API to Duration instead of long ms times2.1.0
KIP-357: Add support to list ACLs per principal2.1.0
KIP-356: Add withCachingDisabled() to StoreBuilder2.1.0
KIP-354: Add a Maximum Log Compaction Lag2.3.0
KIP-353: Improve Kafka Streams Timestamp Synchronization2.1.0
KIP-352: Distinguish URPs caused by reassignment2.5.0 
KIP-351: Add --under-min-isr option to describe topics command2.3.0
KIP-346: Improve LogCleaner behavior on error2.1.0
KIP-345: Introduce static membership protocol to reduce consumer rebalances2.4.0
KIP-342 Add support for custom SASL extensions in OAuthBearer authentication2.1.0
KIP-341: Update Sticky Assignor's User Data Protocol2.3.0
KIP-340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file2.1.0
KIP-339: Create a new IncrementalAlterConfigs API2.3.0 
KIP-338 Support to exclude the internal topics in kafka-topics.sh command2.1.0
KIP-336: Consolidate ExtendedSerializer/Serializer and ExtendedDeserializer/Deserializer2.1.0
KIP-332: Update AclCommand to use AdminClient API2.1.0
KIP-331 Add default implementation to close() and configure() for Serializer, Deserializer and Serde2.3.0
KIP-330: Add retentionPeriod in SessionBytesStoreSupplier2.1.0
KIP-328: Ability to suppress updates for KTables2.1.0 (partially implemented; inactive)
KIP-324: Add method to get metrics() in AdminClient2.1.0
KIP-322: Return new error code for DeleteTopics API when topic deletion disabled.2.1.0
KIP-321: Update TopologyDescription to better represent Source and Sink Nodes2.1.0
KIP-320: Allow fetchers to detect and handle log truncation2.1.0 (partially implemented).  full implementation in 2.4.0
KIP-319: Replace segments with segmentInterval in WindowBytesStoreSupplier2.1.0
KIP-313: Add KStream.flatTransform and KStream.flatTransformValues 

2.2.0 (partially implemented)

/ 2.3.0

KIP-312 Add Overloaded StreamsBuilder Build Method to Accept java.util.Properties2.1.0
KIP-309: Add toUpperCase support to sasl.kerberos.principal.to.local rule2.4.0
KIP-308: Support dynamic update of max.connections.per.ip/max.connections.per.ip.overrides2.1.0
KIP-307: Allow to define custom processor names with KStreams DSL2.3.0 (partially implemented) / 2.4.0
KIP-306: Configuration for Delaying Response to Failed Authentication2.1.0
KIP-305: Add Connect primitive number converters2.0.0
KIP-303: Add Dynamic Routing in Streams Sink2.0.0
KIP-302 - Enable Kafka clients to use all DNS resolved IP addresses2.1.0
KIP-300: Add Windowed KTable API in StreamsBuildernot implemented; inactive
KIP-298: Error Handling in Connect2.0.0
KIP-297: Externalizing Secrets for Connect Configurations2.0.0
KIP-295 Add Streams Configuration Allowing for Optional Topology Optimization2.0.0
KIP-294 - Enable TLS hostname verification by default2.0.0

KIP-291: Separating controller connections and requests from the data plane

2.2.0
KIP-292: Add transformValues() method to KTable2.0.0
KIP-290: Support for Prefixed ACLs2.0.0
KIP-289: Improve the default group id behavior in KafkaConsumer2.2.0
KIP-285: Connect Rest Extension Plugin2.0.0
KIP-284: Set default retention ms for Streams repartition topics to Long.MAX_VALUE2.0.0
KIP-283: Efficient Memory Usage for Down-Conversion2.0.0
KIP-282: Add the listener name to the authentication context2.0.0
KIP-281: ConsumerPerformance: Increase Polling Loop Timeout and Make It Reachable by the End User2.0.0
KIP-280: Enhanced log compactionWIP
KIP-279: Fix log divergence between leader and follower after fast leader fail over2.0.0
KIP-278 - Add version option to Kafka's commands2.0.0
KIP-277 - Fine Grained ACL for CreateTopics API2.0.0
KIP-276 - Add StreamsConfig prefix for different consumers2.0.0
KIP-274: Kafka Streams Skipped Records Metrics2.0.0
KIP-272: Add API version tag to broker's RequestsPerSec metric2.0.0
KIP-270 - A Scala Wrapper Library for Kafka Streams2.0.0
KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade2.0.0
KIP-267: Add Processor Unit Test Support to Kafka Streams Test Utils2.0.0
KIP-266: Fix consumer indefinite blocking behavior2.0.0
KIP-265: Make Windowed Serde to public APIs2.0.0
KIP-261: Add Single Value Fetch in Window Stores2.0.0
KIP-258: Allow to Store Record Timestamps in RocksDB2.3.0 (partially implemented; inactive)
KIP-257 - Configurable Quota Management2.0.0
KIP-255: OAuth Authentication via SASL/OAUTHBEARER2.0.0
KIP-251: Allow timestamp manipulation in Processor API2.0.0
KIP-249: Add Delegation Token Operations to KafkaAdminClient2.0.0
KIP-247: Add public test utils for Kafka Streams1.1.0
KIP-245: Use Properties instead of StreamsConfig in KafkaStreams constructor2.0.0
KIP-244: Add Record Header support to Kafka Streams Processor API2.0.0
KIP-243: Make ProducerConfig and ConsumerConfig constructors public1.1.0
KIP-239 Add queryableStoreName() to GlobalKTable1.1.0
KIP-238: Expose Kafka cluster ID in Connect REST API1.1.0
KIP-237: More Controller Health Metrics2.0.0
KIP-235: Add DNS alias support for secured connection2.1.0
KIP-233: Simplify StreamsBuilder#addGlobalStore1.1.0
KIP-231: Improve the Required ACL of ListGroups API2.1.0
KIP-229: DeleteGroups API1.1.0
KIP-227 - Introduce Incremental FetchRequests to Increase Partition Scalability1.1.0
KIP-226 - Dynamic Broker Configuration1.1.0
KIP-225 - Use tags for consumer “records.lag” metrics1.1.0
KIP-224: Add configuration parameter `retries` to Streams API1.1.0
KIP-223 - Add per-topic min lead and per-partition lead metrics to KafkaConsumer2.0.0
KIP-222 - Add Consumer Group operations to Admin API2.0.0
KIP-221: Enhance DSL with Connecting Topic Creation and Repartition Hint2.6.0
KIP-220: Add AdminClient into Kafka Streams' ClientSupplier1.1.0
KIP-219: Improve quota communication2.0.0
KIP-218: Make KafkaFuture.Function java 8 lambda compatible1.1.0
KIP-216: IQ should throw different exceptions for different errors2.8.0 (WIP)
KIP-215: Add topic regex support for Connect sinks1.1.0
KIP-214: Add zookeeper.max.in.flight.requests config to the broker1.1.0
KIP-213 Support non-key joining in KTable2.4.0
KIP-212: Enforce set of legal characters for connector names1.1.0
KIP-211: Revise Expiration Semantics of Consumer Group Offsets2.1.0
KIP-210 - Provide for custom error handling when Kafka Streams fails to produce1.1.0
KIP-208: Add SSL support to Kafka Connect REST interface1.1.0
KIP-207: Offsets returned by ListOffsetsResponse should be monotonically increasing even during a partition leader change2.2.0
KIP-206: Add support for UUID serialization and deserialization2.1.0
KIP-205: Add all() and range() API to ReadOnlyWindowStore1.1.0
KIP-204 : Adding records deletion operation to the new Admin Client API1.1.0
KIP-203: Add toLowerCase support to sasl.kerberos.principal.to.local rule 1.1.0
KIP-202: Move merge() from StreamsBuilder to KStream1.0.0
KIP-198: Remove ZK dependency from Streams Reset Tool1.0.0
KIP-197 Connect REST API should include the connector type when describing a connector1.0.0
KIP-196: Add metrics to Kafka Connect framework1.0.0
KIP-195: AdminClient.createPartitions1.0.0
KIP-192 : Provide cleaner semantics when idempotence is enabled1.0.0
KIP-191: KafkaConsumer.subscribe() overload that takes just Pattern1.0.0
KIP-190: Handle client-ids consistently between clients and brokers1.0.0
KIP-189: Improve principal builder interface and add support for SASL1.0.0
KIP-188 - Add new metrics to support health checks1.0.0
KIP-187 - Add cumulative count metric for all Kafka rate metrics1.0.0
KIP-186: Increase offsets retention default to 7 days2.0.0
KIP-183 - Change PreferredReplicaLeaderElectionCommand to use AdminClient2.2.0
KIP-182: Reduce Streams DSL overloads and allow easier use of custom storage engines1.0.0
KIP-180: Add a broker metric specifying the number of consumer group rebalances in progress1.1.0
KIP-177: Consumer perf tool should count rebalance time1.0.0
KIP-176: Remove deprecated new-consumer option for tools2.0.0
KIP-175: Additional '--describe' views for ConsumerGroupCommand1.1.0
KIP-174 - Deprecate and remove internal converter configs in WorkerConfig2.0.0
KIP-173: Add prefix to StreamsConfig to enable setting default internal topic configs1.0.0
KIP-171 - Extend Consumer Group Reset Offset for Stream Application1.1.0
KIP-168: Add GlobalTopicCount and GlobalPartitionCount metric per cluster1.0.0
KIP-167: Add interface for the state store restoration process1.0.0

KIP-164 - Add UnderMinIsrPartitionCount and per-partition UnderMinIsr metrics

1.0.0

KIP-163: Lower the Minimum Required ACL Permission of OffsetFetch

1.0.0
KIP-162: Enable topic deletion by default1.0.0
KIP-161: streams deserialization exception handlers1.0.0
KIP-160: Augment KStream.print(), KStream.writeAsText() to allow users pass in extra parameters in the printed string1.0.0
KIP-158: Kafka Connect should allow source connectors to set topic-specific settings for new topics2.6.0
KIP-157 - Add consumer config options to streams reset tool1.0.0
KIP-156 Add option "dry run" to Streams application reset tool0.11.0.0
KIP-155 - Add range scan for windowed state stores0.11.0.0
KIP-154 Add Kafka Connect configuration properties for creating internal topics0.11.0.0
KIP-153: Include only client traffic in BytesOutPerSec metric0.11.0.0
KIP-152 - Improve diagnostics for SASL authentication failures1.0.0
KIP-151 Expose Connector type in REST API0.11.0.0
KIP-150 - Kafka-Streams Cogroup2.5.0
KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner1.1.0 (partially implemented; inactive)
KIP-146 - Classloading Isolation in Connect

0.11.0.0

KIP-145 - Expose Record Headers in Kafka Connect1.1.0
KIP-144: Exponential backoff for broker reconnect attempts0.11.0.0
KIP-143: Controller Health Metrics0.11.0.0
KIP-140: Add administrative RPCs for adding, deleting, and listing ACLs0.11.0.0
KIP-138: Change punctuate semantics1.0.0
KIP-137: Enhance TopicCommand --describe to show topics marked for deletion0.11.0.0
KIP-136: Add Listener name to SelectorMetrics tags0.11.0.0
KIP-134: Delay initial consumer group rebalance0.11.0.0
KIP-133: Describe and Alter Configs Admin APIs0.11.0.0
KIP-131 - Add access to OffsetStorageReader from SourceConnector2.6.0
KIP-130: Expose states of active tasks to KafkaStreams public API1.0.0
KIP-129: Streams Exactly-Once Semantics0.11.0.0
KIP-128: Add ByteArrayConverter for Kafka Connect0.11.0.0
KIP-126 - Allow KafkaProducer to split and resend oversized batches.0.11.0.0
KIP-124 - Request rate quotas0.11.0.0
KIP-123: Allow per stream/table timestamp extractor0.11.0.0
KIP-122: Add Reset Consumer Group Offsets tooling0.11.0.0
KIP-121: Add KStream peek method

0.11.0.0

KIP-120: Cleanup Kafka Streams builder API1.0.0
KIP-119: Drop Support for Scala 2.10 in Kafka 0.110.11.0.0
KIP-118: Drop Support for Java 72.0.0
KIP-117: Add a public AdminClient API for Kafka admin operations0.11.0.0
KIP-115: Enforce offsets.topic.replication.factor upon __consumer_offsets auto topic creation0.11.0.0
KIP-114: KTable state stores and improved semantics0.11.0.0
KIP-113: Support replicas movement between log directories1.1.0
KIP-112: Handle disk failure for JBOD1.0.0
KIP-110: Add Codec for ZStandard Compression2.1.0
KIP-109: Old Consumer Deprecation0.11.0.0
KIP-108: Create Topic Policy0.10.2.0
KIP-107: Add deleteRecordsBefore() API in AdminClient0.11.0.0

KIP-106 - Change Default unclean.leader.election.enabled from True to False

0.11.0.0
KIP-105: Addition of Record Level for Sensors0.10.2.0
KIP-104: Granular Sensors for Streams 0.10.2.0
KIP-103: Separation of Internal and External traffic0.10.2.0
KIP-102 - Add close with timeout for consumers0.10.2.0
KIP-101 - Alter Replication Protocol to use Leader Epoch rather than High Watermark for Truncation0.11.0.0
KIP-100 - Relax Type constraints in Kafka Streams API0.10.2.0
KIP-99: Add Global Tables to Kafka Streams0.10.2.0
KIP-98 - Exactly Once Delivery and Transactional Messaging0.11.0.0
KIP-97: Improved Kafka Client RPC Compatibility Policy0.10.2.0
KIP-96 - Add per partition metrics for in-sync and assigned replica count0.10.2.0
KIP-94 Session Windows0.10.2.0
KIP-93: Improve invalid timestamp handling in Kafka Streams0.10.2.0
KIP-92 - Add per partition lag metrics to KafkaConsumer0.10.2.0
KIP-91 Provide Intuitive User Timeouts in The Producer2.1.0
KIP-90 - Remove zkClient dependency from Streams0.10.2.0
KIP-89: Allow sink connectors to decouple flush and offset commit0.10.2.0
KIP-88: OffsetFetch Protocol Update0.10.2.0
KIP-86: Configurable SASL callback handlers2.0.0
KIP-85: Dynamic JAAS configuration for Kafka clients0.10.2.0
KIP-84: Support SASL SCRAM mechanisms0.10.2.0
KIP-82 - Add Record Headers0.11.0.0
KIP-81: Bound Fetch memory usage in the consumer2.8.0 (WIP)
KIP-79 - ListOffsetRequest/ListOffsetResponse v1 and add timestamp search methods to the new consumer0.10.1.0
KIP-78: Cluster Id0.10.1.0
KIP-77: Improve Kafka Streams Join Semantics0.10.2.0
KIP-75 - Add per-connector Converters0.10.1.0
KIP-74: Add Fetch Response Size Limit in Bytes0.10.1.0
KIP-73: Replication Quotas0.10.1.0
KIP-72: Allow putting a bound on memory consumed by Incoming request 1.0.0
KIP-71: Enable log compaction and deletion to co-exist0.10.1.0
KIP-70: Revise Partition Assignment Semantics on New Consumer's Subscription Change0.10.1.0
KIP-67: Queryable state for Kafka Streams0.10.1.0
KIP-66: Single Message Transforms for Kafka Connect0.10.2.0 / 0.11.0.0
KIP-65: Expose timestamps to Connect0.10.1.0
KIP-63: Unify store and downstream caching in streams0.10.1.0
KIP-62: Allow consumer to send heartbeats from a background thread0.10.1.0
KIP-60 - Make Java client classloading more flexible0.10.1.0
KIP-58 - Make Log Compaction Point Configurable0.10.1.0
KIP-57 - Interoperable LZ4 Framing0.10.0.0
KIP-56: Allow cross origin HTTP requests on all HTTP methods0.10.0.0
KIP-55: Secure Quotas for Authenticated Users0.10.1.0
KIP-54: Sticky Partition Assignment Strategy0.11.0.0
KIP-52: Connector Control APIs0.10.0.0
KIP-51 - List Connectors REST API0.10.0.0
KIP-48 Delegation token support for Kafka1.1.0
KIP-45 - Standardize all client sequence interaction on j.u.Collection.0.10.0.0
KIP-43: Kafka SASL enhancements0.10.0.0
KIP-42: Add Producer and Consumer Interceptors0.10.0.0
KIP-41: Consumer Max Records0.10.0.0
KIP-40: ListGroups and DescribeGroup0.9.0.0
KIP-38: ZooKeeper Authentication0.9.0.0
KIP-36 - Rack aware replica assignment0.10.0.0
KIP-35 - Retrieving protocol version0.10.0.0
KIP-33 - Add a time based log index0.10.1.0
KIP-32 - Add timestamps to Kafka message0.10.0.0
KIP-31 - Move to relative offsets in compressed message sets0.10.0.0
KIP-28 - Add a processor client0.10.0.0
KIP-26 - Add Kafka Connect framework for data import/export0.9.0.0
KIP-25 - System test improvements0.9.0.0
KIP-22 - Expose a Partitioner interface in the new producer0.9.0.0
KIP-21 - Dynamic Configuration0.9.0.0 (WIP)
KIP-20 Enable log preallocate to improve consume performance under windows and some old Linux file system0.9.0.0
KIP-19 - Add a request timeout to NetworkClient0.9.0.0
KIP-16 - Automated Replica Lag Tuning0.9.0.0
KIP-15 - Add a close method with a timeout in the producer0.9.0.0
KIP-13 - Quota Design0.9.0.0
KIP-12 - Kafka Sasl/Kerberos and SSL implementation0.9.0.0
KIP-11 - Kafka Authorizer design0.9.0.0
KIP-8 - Add a flush method to the producer API0.9.0.0
KIP-4 - Metadata Protocol Changes0.10.0.0
KIP-4 - Command line and centralized administrative operations0.9.0.0, 0.10.0.0, 0.10.1.0
KIP-3 - Mirror Maker Enhancement0.9.0.0
KIP-2 - Refactor brokers to allow listening on multiple ports and IPs0.9.0.0
KIP-1 - Remove support of request.required.acks0.9.0.0


KIPs under discussion

Please insert new rows in sorted order (ascending by KIP number). Please move the adopted/accepted KIPs to the "Adopted KIPs" table above.

KIPComment
KIP-59: Proposal for a kafka broker commandSent emails to Dev discussion group. Work tracked under KAFKA-3663.
KIP-125: ZookeeperConsumerConnector to KafkaConsumer Migration and Rollback
KIP-135 : Send of null key to a compacted topic should throw non-retriable error back to user
KIP 141 - ProducerRecord & SourceRecord: Add timestamp constructors
KIP-142: Add ListTopicsRequest to efficiently list all the topics in a cluster
KIP-148: Add a connect timeout for client
KIP-166 - Add a tool to make amounts of replicas and leaders on brokers balanced
KIP-169 - Lag-Aware Partition Assignment Strategy
KIP-178: Size-based log directory selection strategy


KIP-185: Make exactly once in order delivery the default producer setting


KIP-193: Add SchemaBuilder.from(Schema)
KIP-201: Rationalising Policy interfaces

KIP-209 - Connection String Support


KIP-217: Expose a timeout to allow an expired ZK session to be re-created
KIP-228 Negative record timestamp supportVoting in progress
KIP-234: add support for getting topic defaults from AdminClient
KIP-236: Interruptible Partition Reassignment Discussion
KIP-240: AdminClient.listReassignments() AdminClient.describeReassignments()
KIP-242: Mask password in Kafka Connect Rest API response
KIP-250 Add Support for Quorum-based Producer Acknowledgment
KIP-252 - Extend ACLs to allow filtering based on ip ranges and subnets
KIP-253: Support in-order message delivery with partition expansion Discussion
KIP-254: JsonConverter Exception Handeling
KIP-264: Add a consumer metric to record raw fetch sizeVoting in progress
KIP-271: Add NetworkClient redirectorDiscussion
KIP-273: Kafka to support using ETCD beside ZookeeperDiscussion
KIP-293 Add new metrics for consumer/replication fetch requestsVoting in progress
KIP-296: Add connector level configurability for producer/consumer client configs Discussion
KIP-301: Schema Inferencing for JsonConverterDiscussion
KIP-304: Connect runtime mode improvements for container platformsDiscussion
KIP-315: Stream Join Sticky AssignorDiscussion
KIP-316: Command-line overrides for ConnectDistributed worker propertiesDiscussion
KIP-317: Add end-to-end data encryption functionality to Apache KafkaDiscussion
KIP-325: Extend Consumer Group Command to Show Beginning OffsetsVoting in Progress345
KIP-333: Add faster mode of rebalancingDiscussion
KIP-334 - Include partitions in exceptions raised during consumer record deserialization/validation
KIP-348 Eliminate null from SourceTask#poll()
KIP-350: Allow kafka-topics.sh to take brokerid as parameter to show partitions associated with it
KIP-356: Add KafkaConsumer fetch-error-rate and fetch-error-total metricsDiscussion
KIP-363: Allow performance tools to print final results to output fileDiscussion
KIP-370: Remove Orphan PartitionsDiscussion
KIP-375: Kafka Clients - make Metadata#TOPIC_EXPIRY_MS configurableDiscussion
KIP-381: Connect: Tell about records that had their offsets flushed in callback

Voting in progress (restarted 18th January 2019, due to no votes in first attempt)

KIP-384: Add config for incompatible changes to persistent metadataDiscussion
KIP-385: Avoid throwing away prefetched dataDiscussion
KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

Discussion

KIP-388: Add observer interface to record request and responseDiscussion
KIP-395: Encypt-then-MAC Delegation token metadata
KIP-400: Improve exit status in case of errors in ConsoleProducerDiscussion
KIP-403: Increase ProducerPerformance precision by using nanoTimeDraft
KIP-406: GlobalStreamThread should honor custom offset policy.Blocked
KIP-407: Kafka Connect support override worker kafka api configuration with connector configuration that post by rest api

KIP-409: Allow creating under-replicated topics and partitions


KIP-410: Add metric for request handler thread pool utilization by request type

Discussion
KIP-416: Notify SourceTask of ACK'd offsets, metadataDiscussion
KIP-419: Safely notify Kafka Connect SourceTask is stoppedVoting in progress, JIRA exists with pull request
KIP-423: Add JoinReason to Consumer Join Group Protocol

Discussion

KIP-424: Allow suppression of intermediate events based on wall clock timeDiscussion
KIP-426: Persist Broker Id to ZookeeperDiscussion
KIP-435: Internal Partition Reassignment BatchingDiscussion
KIP-438: Expose task, connector IDs in Connect APIDiscussion
KIP-439: Cleanup built-in Store interfacesDiscussion
KIP-448: Add State Stores Unit Test Support to Kafka Streams Test UtilsDiscussion
KIP-452: Tool to view cluster statusDiscussion, JIRA exists
KIP-468: Avoid decompression of record when validate record at server in the scene of inPlaceAssignment .Under discussion
KIP-473: Enable KafkaLog4JAppender to use SASL Authentication Callback HandlersUnder Discussion 
KIP-477: Add PATCH method for connector config in Connect REST APIUnder Discussion
KIP-487: Automatic Topic Creation on ProducerUnder Discussion
KIP-489: Kafka Consumer Record Latency MetricUnder Discussion
KIP-490: New metric to count offsets expired without being consumed by a consumer groupUnder discussion
KIP-491: Preferred Leader Deprioritized List (Temporary Blacklist)Under discussion
KIP-494: Connect REST Endpoint to get Transformations (SMTs)
KIP-498: Add client-side configuration for maximum response size to protect against OOMSent emails to Dev discussion group. Work tracked under KAFKA-4090.
KIP-502: Connect SinkTask.put(...) to specify ArrayList<SinkRecord> in Signature
KIP-505: Add new public method to only update assignment metadata in consumerUnder discussion
KIP-506: Allow setting SCRAM password via Admin interfaceUnder discussion
KIP-508: Make Suppression State QueriableUnder discussion
KIP-509: Rebalance and restart ProducersUnder discussion
KIP-510: Metrics library upgradeUnder discussion
KIP-512: Adding headers to RecordMetaDataSent mail to dev discussion group
KIP-513: Distinguish between Key and Value serdes in scala wrapper library for kafka streamsSent mail to dev discussion group
KIP-536: Propagate broker start time to Admin APIUnder discussion
KIP-539: Add mechanism to flush records out in low volume suppression buffersUnder discussion.
KIP-540: Implement per key stream time trackingDraft (In Progress)
KIP-542: Partition Reassignment ThrottlingDraft
KIP-547: Extend ConsumerInterceptor to allow modification of Consumer CommitsUnder Discussion
KIP-548 Add Option to enforce rack-aware custom partition reassignment executionUnder Discussion
KIP-549: Surface Consumer's Metadata in KafkaAdminClient#describeConsumerGroupsUnder Discussion
KIP-550: Mechanism to Delete Stray Partitions on BrokerDraft
KIP-552: Suppress "unused config" warning

Under Discussion

KIP-560: Auto infer external topic partitions in stream reset tool

Under Discussion

KIP-561: Regex Support for ConsumerGroupCommand

Under Discussion
KIP-563: Add 'tail -n' feature for ConsoleConsumerUnder Discussion
KIP-564: Add new cached authorizer:change the dim of cacheUnder Discussion
KIP-565: Using AclCommand,avoid call the global method loadcache in SimpleAclAuthorizerUnder Discussion
KIP-566: Add rebalance callbacks to ConsumerInterceptorUnder Discussion
KIP-567: Kafka Cluster Audit

Under Discussion

KIP-575: build a Kafka-Exporter by JavaUnder Discussion
KIP-576: Support dynamic update of more broker configs related to replicationUnder Discussion
KIP-578: Add configuration to limit number of partitionsVoting
KIP-579: new exception on min.insync.replicas > replication.factorVoting in progress
KIP-582: Add a "continue" option for Kafka Connect error handlingUnder Discussion
KIP-583: add tag "partition" to BrokerTopicMetrics so as to observe the partition metrics on the same brokerUnder Discussion
KIP-587: Suppress detailed responses for handled exceptions in security-sensitive environmentsUnder Discussion
KIP-592: Replicate mirrormaker topics from earliestUnder Discussion
KIP-595: A Raft Protocol for the Metadata QuorumUnder Discussion
KIP-596: Safely abort Producer transactions during application shutdownUnder Discussion
KIP-598: Augment TopologyDescription with store and source / sink serde informationUnder Discussion
KIP-619: Add internal topic creation supportUnder Discussion
KIP-625: Richer encodings for integral-typed protocol fieldsUnder Discussion

KIP-628: ConsumerPerformance's multi-thread implementation

Sent email to Dev discussion group. Work tracked under KAFKA-10136.
KIP-636: Make RPC error codes and messages tagged fieldsUnder Discussion
KIP-637: Include min.insync.replicas in MetadataResponse to make Producer smarter in partitioning eventsDraft
KIP-639: Move nodeLevelSensor and storeLevelSensor methods from StreamsMetricsImpl to StreamsMetricsUnder Discussion
KIP-640: Add log compression analysis toolUnder Discussion
KIP-649: Dynamic client configurationUnder Discussion
KIP-655: Windowed Distinct Operation for Kafka Streams APIVoting in progress
KIP-656: MirrorMaker2 Exactly-once SemanticsDraft
KIP-665: Kafka Connect Hash SMT

Voting in progress

KIP-668: Expose REST endpoint to list converter pluginsUnder Discussion
KIP-669 Preserve Source Partition in Kafka Streams from contextDraft

KIP-674: API to Aggregate Metrics in Kafka Streams

Under Discussion

KIP-675: Convert a KTable to a KStream using the previous valueUnder Discussion
KIP-678: New Kafka Connect SMT for plainText => struct with RegexVoting
KIP-681: Rename master key in delegation token featureDraft
KIP-682: Connect TimestampConverter support for multiple fields and multiple input formatsUnder Discussion
KIP-683: Add recursive support to Connect Cast and ReplaceField transforms, and support for casting complex types to either a native or JSON stringUnder Discussion
KIP-685: Loosen permission for listing reassignmentsUnder Discussion
KIP-686: API to ensure Records policy on the brokerUnder Discussion
KIP-687: Automatic Reloading of Security StoreUnder Discussion
KIP-688: Support dynamic update of delete.topic.enable config

Under Discussion

KIP-692: Make AdminClient value object constructors publicUnder discussion

KIP-693: Client-side Circuit Breaker for Partition Write Errors

Voting in progress

KIP-694: Support Reducing Partitions for Topics

Under Discussion
KIP-697: Stricter parsing of addresses in configsUnder Discussion
KIP-701: Add --override option to bin/connect-distributed.shUnder Discussion
KIP-702: The control plane needs to force the validation of requests from the controllerUnder Discussion
KIP-703: Add a metric for reporting idle connections closed

Under Discussion

KIP-705: Selectively Disable Topology OptimizationsDraft
KIP-706: Add method "Producer#produce" to return CompletionStage instead of FutureUnder Discussion
KIP-711 Deprecate org.apache.kafka.streams.errors.BrokerNotFoundExceptionUnder Discussion
KIP-712: Shallow MirroringUnder Discussion
KIP-713: Validation of Enums in configurationUnder Discussion
KIP-718: Make KTable Join on Foreign key unopinionatedUnder Discussion
KIP-723: Add socket.tcp.no.delay property to Kafka ConfigUnder Discussion

KIP-727: Add --under-preferred-replica-partitions option to describe topics command

Under Discussion

KIP-729: Custom validation of records on the broker prior to log append

Under Discussion
KIP-731: Record Rate Limiting for Kafka ConnectUnder Discussion
KIP-736: Report the true end to end fetch latencyUnder Discussion
KIP-737: Add canTrackSource to ReplicationPolicyUnder Discussion
KIP-739: Block Less on KafkaProducer#sendUnder Discussion
KIP-749: Add --files and --file-separator options to the ConsoleProducerUnder Discussion
KIP-752: Support --bootstrap-server in ReplicaVerificationToolUnder Discussion
KIP-754: Make Scala case class's finalUnder Discussion
KIP-755: Add new AUTO_CREATE ACL for auto topic creationUnder Discussion
KIP-759: Unneeded repartition cancelingAccepted, targeting 3.7
KIP-760: Minimum value for segment.ms and segment.bytesUnder Discussion
KIP-762: Delete Committed Connect RecordsUnder Discussion
KIP-767: Connect Latency MetricsUnder Discussion
KIP-774: Deprecate public access to Admin client's *Result constructors

Under Discussion

KIP-777: Improved testability for Admin clientDraft
KIP-780: Support fine-grained compression optionsUnder Discussion
KIP-781: Improve MirrorMaker2's client configurationUnder Discussion
KIP-782: Expandable batch size in producerVoting
KIP-785: Automatic storage formattingUnder Discussion
KIP-786: Emit Metric Client Quota ValuesUnder Discussion 
KIP-789: Use localhost:9092 as default bootstrap-server/broker-list in client toolsKIP page WIP
KIP-799: Align behaviour for producer callbacks with documented behaviourUnder Discussion

KIP-802: Validation Support for Kafka Connect SMT Options

Under Discussion

KIP-803: Add Task ID and Connector Name to Connect Task ContextUnder Discussion
KIP-804: OfflinePartitionsCount Tagged by TopicUnder Discussion
KIP-807: Refactor KafkaStreams exposed metadata hierarchyUnder Discussion
KIP-809: Support live upgrades with dynamic addition/removal of modular topologiesKIP page WIP
KIP-816: Topology changes without local state resetUnder Discussion
KIP-817: Fix inconsistency in dynamic application log levelsUnder Discussion
KIP-819: Merge multiple KStreams in one operationUnder Discussion
KIP-822: Optimize the semantics of KafkaConsumer#pause to be consistent between the two RebalanceProtocolsUnder Discussion
KIP-823: Update Admin::describeConfigs to allow fetching specific configurationsKIP page WIP

KIP-826: Define platforms supported

Under Discussion
KIP-828: Add the corresponding validator to the configuration where the validator is missingUnder Discussion
KIP-829: (console-consumer) add print.topic propertyUnder Discussion
KIP-838 Simulate batching and compressionUnder Discussion

KIP-839: Provide builders for KafkaProducer/KafkaConsumer and KafkaStreams

Under Discussion
KIP-842: Add richer group offset reset mechanismsUnder Discussion
KIP-849: Expose logdirs total and usable space via kafka-log-dirs.shUnder Discussion
KIP-853: KRaft Voter ChangesUnder Discussion
KIP-855: Add schema.namespace parameter to SetSchemaMetadata SMT in Kafka ConnectUnder Discussion
KIP-856: KRaft Disk Failure RecoveryDraft
KIP-857: Streaming recursion in Kafka StreamsUnder Discussion

KIP-860: Add client-provided option to guard against replication factor change during partition reassignments

Under Discussion
KIP-864: Add End-To-End Latency Metrics to ConnectorsUnder Discussion
KIP-870: Retention policy based on record event timeUnder Discussion
KIP-873: ExceptionHandlingDeserializer, RetryDeserializer, PipeSerializer, PipeDeserializerUnder Discussion

KIP-877: Mechanism for plugins and connectors to register metrics

Under Discussion
KIP-879: Multi-level Rack AwarenessUnder Discussion
KIP-880: X509 SAN based SPIFFE URI ACL within mTLS Client CertificatesUnder Discussion
KIP-882: Kafka Connect REST API configuration validation timeout improvementsUnder Discussion

KIP-883: Add isDeleted flag when stopping a connector

Under Discussion
KIP-885: Expose Broker's Name and Version to ClientsUnder Discussion
KIP-886: Add Client Producer and Consumer BuildersUnder Discussion

KIP-888: Batch describe ACLs and describe client quotas

Under Discussion

KIP-891: Running multiple versions of a connectorUnder Discussion
KIP-895: Dynamically refresh partition count of __consumer_offsetsUnder Discussion
KIP-899: Allow clients to rebootstrap

Under Discussion

KIP-901: Add connectorDeleted flag when stopping tasks

Under Discussion

KIP-905: Broker interceptors

Under Discussion

KIP-910: Update Source offsets for Source Connectors without producing records

Under Discussion

KIP-912: Support decreasing send's block time without worrying about metadata's fetch 

Under Discussion

KIP-913: add new method to provide possibility for accelerate first record's sending

Under Discussion

KIP-916: MM2 distributed mode flow log context

Under Discussion
KIP-917: Additional custom metadata for remote log segmentUnder Discussion

KIP-918: MM2 Topic And Group Listener

Under Discussion

KIP-921: OpenJDK CRaC support

Under Discussion

KIP-922: Add the traffic metric of the partition dimension

Under Discussion

KIP-924: Introduce public config for Streams task assignor

Under Discussion

KIP-926: introducing acks=min.insync.replicas config

Under Discussion

KIP-928: Making Kafka resilient to log directories becoming full

Under Discussion

KIP-930: Rename ambiguous Tiered Storage Metrics

Under Discussion

KIP-931: Flag to ignore unused message attribute field

Under Discussion

KIP-932: Queues for Kafka

Under Discussion

KIP-933: Publish metrics when source connector fails to poll data

Under Discussion

KIP-934: Add DeleteTopicPolicy

Under Discussion

KIP-935: Extend AlterConfigPolicy with existing configurations

Under Discussion

KIP-936: Throttle number of active PIDs

Under Discussion

KIP-939: Support Participation in 2PC

Under Discussion

KIP-940: Broker extension point for validating record contents at produce time

Under Discussion

KIP-943: Add independent “offset.storage.segment.bytes” for connect-distributed.properties

Under Discussion

KIP-945: Update threading model for ConsumerWIP
KIP-946: Modify exceptions thrown by Consumer APIsWIP

KIP-948: Allow custom prefix for internal topic names in Kafka Streams

Under Discussion

KIP-950: Tiered Storage Disablement

Under Discussion

KIP-952: Regenerate segment-aligned producer snapshots when upgrading to a Kafka version supporting Tiered Storage

Under Discussion

KIP-953: partition method to be overloaded to accept headers as well.

Draft

KIP-955: Add stream-table join on foreign key

Under Discussion

KIP-964 Have visibility when produce requests become "async"   

Under discussion

KIP-965: Support disaster recovery between clusters by MirrorMaker

Under Discussion

KIP-969: Support range Interactive Queries (IQv2) for Versioned State Stores 


KIP-971: Expose replication-record-lag MirrorMaker2 metric

KIP-972: Add the metric of the current running version of kafka

Under Discussion

KIP-973: Expose per topic replication rate metrics

Under Discussion

KIP-974: Docker Image for GraalVM based Native Kafka Broker

Under Discussion

KIP-981: Manage Connect topics with custom implementation of Admin

Under Discussion

KIP-982: Access SslPrincipalMapper and kerberosShortNamer in Custom KafkaPrincipalBuilder

Voting in progress

(PR is reviewed by Contributors)

KIP-983: Full speed async processing during rebalance

Draft

KIP-984: Add pluggable compression interface to Kafka

Under Discussion

KIP-986: Cross-Cluster Replication

Draft

KIP-987: Connect Static Assignments

Under Discussion

KIP-989: RocksDB Iterator Metrics

Under Discussion

KIP-990: Capability to PAUSE Tasks on DeserializationException

Under Discussion

KIP-991: Allow DropHeaders SMT to drop headers by wildcard/regexp

Under Discussion

KIP-995: Allow users to specify initial offsets while creating connectors

Under Discussion

KIP-997: update WindowRangeQuery and unify WindowKeyQuery and WindowRangeQuery

Under Discussion 
Vote

KIP-999: Server-side Consumer Lag Metrics

Draft

KIP-1002: Fetch remote segment indexes at once

Under Discussion

KIP-1003: Signal next segment when remote fetching

Under Discussion

KIP-1006: Remove SecurityManager Support

Draft

KIP-1008: ParKa - the Marriage of Parquet and Kafka

Under Discussion

KIP-1009: Add Broker-level Throttle Configurations

Under Discussion

KIP-1010: Topic Partition Quota

Under Discussion

KIP-1011: Use incrementalAlterConfigs when updating broker configs by kafka-configs.sh

Under Discussion

KIP-1015: Limit number of ssl connections in brokers

Under Discussion

KIP-1016 Make MM2 heartbeats topic name configurable

Under Discussion

KIP-1017: Health check endpoint for Kafka Connect

Under Discussion

KIP-1018: Introduce max remote fetch timeout config for DelayedRemoteFetch requests

Under Discussion

KIP-1021: Allow to get last stable offset (LSO) in kafka-get-offsets.sh

Under Discussion

KIP-1022: Formatting and Updating Features

Under Discussion

KIP-1023: Follower fetch from tiered offset

Under Discussion

KIP-1027: Add MockFixedKeyProcessorContext

Under Discussion

KIP-1028: Docker Official Image for Apache Kafka

Under Discussion

KIP-1031: Control offset translation in MirrorSourceConnector

Under Discussion

KIP-1032: Upgrade to Jakarta and JavaEE 9 in Kafka 4.0

Under Discussion


Go to link

Dormant/inactive KIPs

Please insert new rows in sorted order (ascending by KIP number).

KIPComment
KIP-6 - New reassignment partition logic for rebalancingNeeds more detail
KIP-10 - Running Producer, Consumers and Brokers on Mesos
KIP-14 - Tools standardization
KIP-17 - Add HighwaterMarkOffset to OffsetFetchResponse
KIP-23 - Add JSON/CSV output and looping options to ConsumerGroupCommand
KIP-27 - Conditional Publish
KIP-30 - Allow for brokers to have plug-able consensus and meta data storage sub systems
KIP-37 - Add Namespaces to Kafka
KIP-39: Pinning controller to broker
KIP-44 - Allow Kafka to have a customized security protocol
KIP-46 - Self Healing
KIP-47 - Add timestamp-based log deletion policy
KIP-49 - Fair Partition Assignment Strategy
KIP-53 - Add custom policies for reconnect attempts to NetworkdClient
KIP-59: Proposal for a kafka broker command
KIP-64 -Allow underlying distributed filesystem to take over replication depending on configuration
KIP-68 Add a consumed log retention before log retention
KIP-69 - Kafka Schema RegistryDraft
KIP-76 Enable getting password from executable rather than passing as plaintext in config files
KIP-87 - Add Compaction Tombstone FlagPossibly displaced by KIP-82
KIP-95: Incremental Batch Processing for Kafka Streams
KIP-159: Introducing Rich functions to Streams
KIP-260: add primary join operation for Stream-Stream join (WIP)Draft

KIP-269: Substitution Within Configuration Values


KIP-311: Async processing with dynamic scheduling in Kafka Streams
KIP-314: KTable to GlobalKTable Bi-directional Join
KIP-326: Schedulable KTable as Graph source
KIP-349: Priorities for Source Topics

Functionality can be realized with existing API although at a lower level. 

KIP-335: Consider configurations for KafkaStreams
KIP-362: Support Dynamic Session Window
KIP-378: Enable Dependency Injection for Kafka Streams handlers
KIP-408: Add asynchronous processing to Kafka Streams
KIP-422: Add support for client quota configuration in the Kafka Admin ClientKIP-546 added the necessary APIs instead
KIP-457: Add DISCONNECTED status to Kafka Streams
KIP-459: Improve KafkaStreams#close
KIP-463: Auto-configure non-default Serdes passed alongside the TopologyBuilder
KIP-611: Improved Handling of Abandoned Connectors and Tasks
KIP-645: Replace Windows with a proper interfaceStill nice to have, just not as important as I thought.
KIP-634: Complementary support for headers and record metadata in Kafka Streams DSLKIP-820 offers a workaround, and potential DSL v2 could replace the proposal.

Discarded KIPs

Please insert new rows in sorted order (ascending by KIP number).

KIPComment
KIP-5 - Broker Configuration ManagementSuperseded by KIP-21
KIP-7 - Security - IP Filtering
KIP-9 - SSL SupportAdopted via KAFKA-1690 but not via this KIP process
KIP-18 - JBOD SupportSuperseded by KIP-112 and KIP-113 
KIP-24 - Remove ISR information from TopicMetadataRequest and add broker level metadata request
KIP-29 - Add an IsrPropagateIntervalMs configuration to KafkaConfigNo longer need after KAFKA-2722

KIP-34 Add Partitioner Change Listener to Partitioner Interface for Multiple Use Case

Per feedback it is better do this parition() method and avoid Thread Coordination etc.
KIP-50 - Move Authorizer to o.a.k.common packageSuperseded by KIP-504
KIP-80: Kafka Rest Server
KIP-83 - Allow multiple SASL authenticated Java clients in a single JVM processAfter KIP-85, no interface changes, reduced to KAFKA-4180
KIP-111: Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.Covered by KIP-189
KIP-116: Add State Store Checkpoint Interval ConfigurationCurrently not needed as checkpointing can be done on commit interval.
KIP-127: Pluggable JAAS LoginModule configuration for SSLSimilar functionality can be implemented using a custom PrincipalBuilder.
KIP-132: Augment KStream.print to allow extra parameters in the printed string

Duplicated by KIP-160

KIP-147: Add missing type parameters to StateStoreSupplier factories and KGroupedStream/Table methodsCovered by KIP-182
KIP-165: Extend Interactive Queries for return latest update timestamp per keyCovered by KIP-258
KIP-170: Enhanced TopicCreatePolicy and introduction of TopicDeletePolicySuperseded by KIP-201: Rationalising Policy interfaces
KIP 172: Add regular-expression topic support for sink connectorSuperseded by KIP-215: Add topic regex support for Connect sinks
KIP-179: Change ReassignPartitionsCommand to use AdminClient
KIP-184: Rename LogCleaner and related classes to LogCompactorA lot of configuration change will cause more trouble and doesn't seems to be worth it
KIP-199: Add Kafka Connect offset toolSuperseded by KIP-875: First-class offsets support in Kafka Connect
KIP 230: Name Windowing JoinsCovered by KIP-372: Naming Repartition Topics for Joins and Grouping
KIP-232: Detect outdated metadata using per-partition leaderEpoch fieldMerged into KIP-320: Allow fetchers to detect and handle log truncation
KIP-246: Connect producers and consumers should inherit worker configsWithdrawn because proposal could not be made backward compatible with existing behavior
KIP-248 - Create New ConfigCommand That Uses The New AdminClientDiscarded by author as it's partly covered in KIP-339 and KIP-422
KIP-259: Improve Streams DSL Timestamp Propagation SemanticsChange did not require a KIP.
KIP-262: Metadata should include number of state stores for taskRejected since metadata is no longer required.
KIP-263: Allow broker to skip sanity check of inactive segments on broker startupWithdrawn because the solution we agreed on does not require interface change
KIP-275 - Indicate "isClosing" in the SinkTaskContext
KIP-286: producer.send() should not block on metadata updateWithdrawn because the benefit of not having to wait for metadata is probably not worth the complexity added in producer.
KIP-288: [DISCARDED] Consumer.poll() timeout semantic change and new waitForAssignment methodDiscarded in deference to KIP-266
KIP-310: Add a Kafka Source Connector to Kafka ConnectWithdrawn in favor of KIP-382.
KIP-327: Add describe all topics API to AdminClientWithdrawn in-favor of filter support in Metadata API and KIP-142
KIP-344: The auto-generated client id should be passed to MetricsReporterBug fix so KIP not needed
KIP-347: Enable batching in FindCoordinatorRequestReplaced by KIP-699
KIP-364: Remove implicit Materialized, Consumed and ProducedSubsumed by KIP-365
KIP-383:  Pluggable interface for SSL FactoryReplaced by KIP-519
KIP-391: Allow Producing with Offsets for Cluster ReplicationNot enough interest in favour.
KIP-398: Support reading trust store from classpath

Not enough interest in favour. Can be implemented with mechanism in 

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-8191

KIP-404: Add Kafka Connect configuration parameter for disabling WADL output on OPTIONS request

Discarded, reported as a bug by

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-7759

KIP-432: Additional Broker-Side Opt-In for Default, Unsecure SASL/OAUTHBEARER Implementation

A "security sanity check tool" is a more generic and appropriate solution.
KIP-451: Make TopologyTestDriver output iterable

Discarded in favor of KIP-456

KIP-456: Helper classes to make it simpler to write test logic with TopologyTestDriverDiscarded in favor of KIP-470
KIP-472: Add header to RecordContext/ProducerRecordDiscarded due to change being far more complex than necessary for a simple bug
KIP-485: Make topic optional when using through() operations in DSLMoved to KIP-221
KIP-486: Support custom way to load KeyStore and TrustStoreDiscarded in favor of KIP-519
KIP-522: Update BrokerApiVersionsCommand to use AdminClientDiscarded as this KIP is likely to be misused
KIP-593: Enable --if-exists and --if-not-exists for AdminClient in TopicCommandSuperseded by KIP-604
KIP-615: add ConstrainedCooperativeStickyAssignorDiscarded in favor of modifying existing assignor
KIP-638: Deprecate DescribeLogDirsResponse.[LogDirInfo, ReplicaInfo]

Duplication of KIP-621

KIP-660: Pluggable ReplicaPlacer


KIP-667: Remove deprecated methods from ReadOnlyWindowStoreNo KIP needed
KIP-672: Introduce Kafka Streams Specific Uncaught Exception Handler

Combined with KIP-671

[DISCARD] KIP-658 Deprecate all setters of Headers
KIP-818: Introduce cache-size-bytes-total Task Level MetricDecided to add the new metric in KIP-770.
KIP-717: Deprecate batch-size config from console producerChange did not require a KIP
KIP-756: Move StreamsResetter tool outside of coreSubsumed by KIP-906

KIP-795: Add public APIs for AbstractCoordinator

Discarded in favor of KIP-848
KIP-832: Allow creating a producer/consumer using a producer/consumer configSuperseded by KIP-839
KIP-844: Transactional State StoresDiscarded in favor of KIP-892
KIP-845: 'HasField' predicate for kafka connectDiscarded in favor of improving existing SMTs
KIP 850: REST API for filtering Connector plugins by typeDiscarded as JIRA was obsolete
KIP-871: Fix ByteBufferSerializer#serialize(String, ByteBuffer) compatible problemDiscard as it doesn't change any public API.
KIP-874: TopicRoundRobinAssignorDiscard as it is too specific.
KIP-897: Publish a single kafka (aka core) Maven artifact in Apache Kafka 4.0Taking a different approach.
KIP-908: Add description field to connector configurationDiscarded as the benefits are too limited
KIP-947: Add support for consumer group protocol in ConsumerChange did not require a KIP
KIP-944: Support async runtimes in consumer and KIP-957: Support Async runtimesWithdrawn because the committers do not seem to be convinced that you cannot control on what thread code runs with an async runtime

Discarded KIPs

Please insert new rows in sorted order (by KIP number).

Discarded due to change being far more complex than necessary for a simple bug
KIPComment
KIP-5 - Broker Configuration ManagementSuperseded by KIP-21
KIP-7 - Security - IP Filtering
KIP-9 - SSL SupportAdopted via KAFKA-1690 but not via this KIP process
KIP-18 - JBOD SupportSuperseded by KIP-112 and KIP-113 
KIP-24 - Remove ISR information from TopicMetadataRequest and add broker level metadata request
KIP-29 - Add an IsrPropagateIntervalMs configuration to KafkaConfigNo longer need after KAFKA-2722

KIP-34 Add Partitioner Change Listener to Partitioner Interface for Multiple Use Case

Per feedback it is better do this parition() method and avoid Thread Coordination etc.
KIP-80: Kafka Rest Server
KIP-83 - Allow multiple SASL authenticated Java clients in a single JVM processAfter KIP-85, no interface changes, reduced to KAFKA-4180
KIP-111: Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.Covered by KIP-189
KIP-116: Add State Store Checkpoint Interval ConfigurationCurrently not needed as checkpointing can be done on commit interval.
KIP-127: Pluggable JAAS LoginModule configuration for SSLSimilar functionality can be implemented using a custom PrincipalBuilder.
KIP-132: Augment KStream.print to allow extra parameters in the printed string

Duplicated by KIP-160

KIP-147: Add missing type parameters to StateStoreSupplier factories and KGroupedStream/Table methodsCovered by KIP-182
KIP-165: Extend Interactive Queries for return latest update timestamp per keyCovered by KIP-258
KIP-170: Enhanced TopicCreatePolicy and introduction of TopicDeletePolicySuperseded by KIP-201: Rationalising Policy interfaces
KIP 172: Add regular-expression topic support for sink connectorSuperseded by KIP-215: Add topic regex support for Connect sinks
KIP-179: Change ReassignPartitionsCommand to use AdminClient
KIP-184: Rename LogCleaner and related classes to LogCompactorA lot of configuration change will cause more trouble and doesn't seems to be worth it
KIP 230: Name Windowing JoinsCovered by KIP-372: Naming Repartition Topics for Joins and Grouping
KIP-232: Detect outdated metadata using per-partition leaderEpoch fieldMerged into KIP-320: Allow fetchers to detect and handle log truncation
KIP-246: Connect producers and consumers should inherit worker configsWithdrawn because proposal could not be made backward compatible with existing behavior
KIP-248 - Create New ConfigCommand That Uses The New AdminClientDiscarded by author as it's partly covered in KIP-339 and KIP-422
KIP-262: Metadata should include number of state stores for taskRejected since metadata is no longer required.
KIP-263: Allow broker to skip sanity check of inactive segments on broker startupWithdrawn because the solution we agreed on does not require interface change
KIP-286: producer.send() should not block on metadata updateWithdrawn because the benefit of not having to wait for metadata is probably not worth the complexity added in producer.
KIP-288: [DISCARDED] Consumer.poll() timeout semantic change and new waitForAssignment methodDiscarded in deference to KIP-266
KIP-310: Add a Kafka Source Connector to Kafka ConnectWithdrawn in favor of KIP-382.
KIP-327: Add describe all topics API to AdminClientWithdrawn in-favor of filter support in Metadata API and KIP-142
KIP-344: The auto-generated client id should be passed to MetricsReporterBug fix so KIP not needed
KIP-364: Remove implicit Materialized, Consumed and ProducedSubsumed by KIP-365
KIP-404: Add Kafka Connect configuration parameter for disabling WADL output on OPTIONS request

Discarded, reported as a bug by

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-7759

KIP-432: Additional Broker-Side Opt-In for Default, Unsecure SASL/OAUTHBEARER Implementation

A "security sanity check tool" is a more generic and appropriate solution.
KIP-451: Make TopologyTestDriver output iterable

Discarded in favor of KIP-456

KIP-472: Add header to RecordContext/ProducerRecord

KIP Discussion Recordings

...