Page History

This page is an overview of all adopted and discard KIPs. For current KIP discussions go the parent page.

Adopted

• Kafka 3.8:
  • KIP-813: Shareable State Stores
  • KIP-924: customizable task assignment for Streams
  • KIP-989: Improved StateStore Iterator metrics for detecting leaks
• Kafka 3.7:
  • KIP-714: Client metrics and observability (includes some minor Kafka Streams changes)
  • KIP-925: Rack aware task assignment in Kafka Streams (completed; partially implemented in 3.6)
  • KIP-954: expand default DSL store configuration to custom types
  • KIP-960: Support single-key_single-timestamp interactive queries for versioned state stores
  • KIP-962: Relax non-null key requirement in Kafka Streams
  • KIP-968: Support single-key_multi-timestamp Interactive Queries (IQv2) for Versioned State Stores 
  • KIP-985: Add reverseRange and reverseAll query over kv-store in IQv2
  • KIP-988: Streams Standby Update Listener
  • KIP-992: Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery
• Kafka 3.6:
  • KIP-923: Add A Grace Period to Stream Table Join
  • KIP-925: Rack aware task assignment in Kafka Streams (partially implemented)
  • KIP-941: Range queries to accept null lower and upper bounds
• Kafka 3.5:
  • KIP-399: Extend ProductionExceptionHandler to cover serialization exceptions
  • KIP-869: Improve Streams State Restoration Visibility (partially implemented)
  • KIP-884: Add config to configure KafkaClientSupplier in Kafka Streams
  • KIP-889: Versioned State Stores
  • KIP-904: Kafka Streams - Guarantee subtractor is called before adder if key has not changed
  • KIP-907: Add Boolean Serde to public interface
  • KIP-914: DSL Processor Semantics for Versioned Stores
• Kafka 3.4:
  • KIP-770: Replace "buffered.records.per.partition" & "cache.max.bytes.buffering" with "{statestore.cache}/{input.buffer}.max.bytes" (partially implemented)
  • KIP-837: Allow MultiCasting a Result Record.
  • KIP-862: Self-join optimization for stream-stream joins
  • KIP-865: Support --bootstrap-server in kafka-streams-application-reset
• Kafka 3.3:
  • KIP-812: Introduce another form of the `KafkaStreams.close()` API that forces the member to leave the consumer group
  • KIP-820: Extend KStream process with new Processor API
  • KIP-825: introduce a new API to control when aggregated results are produced
  • KIP-834: Pause / Resume KafkaStreams Topologies
  • KIP-846: Source/sink node metrics for Consumed/Produced throughput in Streams
• Kafka 3.2:
  • KIP-471: Expose RocksDB Metrics in Kafka Streams (completed; partially implemented in v2.4)
  • KIP-591: Add Kafka Streams config to set default state store
  • KIP-708: Rack aware StandbyTask assignment for Kafka Streams
  • KIP-791: Add Record Metadata to State Store Context
  • KIP-796: Interactive Query v2
  • KIP-805: Add range and scan query over kv-store in IQv2
  • KIP-806: Add session and window query over kv-store in IQv2
  • KIP-811: Add config repartition.purge.interval.ms to Kafka Streams
• Kafka 3.1:
  • KIP-633: Deprecate 24-hour Default Grace Period for Windowed Operations in Streams (completed; partially implemented in v3.0)
  • KIP-761: Add Total Blocked Time Metric to Streams
  • KIP-763: Range queries with open endpoints
  • KIP-766: fetch/findSessions queries with open endpoints for SessionStore/WindowStore
  • KIP-775: Custom partitioners in foreign key joins
  • KIP-783: Add TaskId field to StreamsException
• Kafka 3.0:
  
  • KIP-622: Add currentSystemTimeMs and currentStreamTimeMs to ProcessorContext
  • KIP-623: Add "internal-topics" option to streams application reset tool
  • KIP-633: Deprecate 24-hour Default Grace Period for Windowed Operations in Streams (partially implemented; completed in v3.1)
  • KIP-666: Add Instant-based methods to ReadOnlySessionStore
  • KIP-715: Expose Committed offset in streams
  • KIP-732: Deprecate eos-alpha and replace eos-beta with eos-v2
  • KIP-733: change Kafka Streams default replication factor config
  • KIP-741: Change default serde to be null
  • KIP-466: Add support for List<T> serialization and deserialization
  • KIP-695: Further Improve Kafka Streams Timestamp Synchronization
  • KIP-725: Streamlining configurations for WindowedSerializer and WindowedDeserializer.
  • KIP-740: Clean up public API in TaskId and fix TaskMetadata#taskId()
  • KIP-743: Remove config value 0.10.0-2.4 of Streams built-in metrics version config
  • KIP-744: Migrate TaskMetadata and ThreadMetadata to an interface with internal implementation
• Kafka 2.8:
  • KIP-418: A method-chaining way to branch KStream
  • KIP-572: Improve timeouts and retries in Kafka Streams (completed; partially implemented in v2.7)
  • KIP-614: Add Prefix Scan support for State Stores
  • KIP-659: Improve TimeWindowedDeserializer and TimeWindowedSerde to handle window size
  • KIP-663: API to Start and Shut Down Stream Threads
  • KIP-696: Update Streams FSM to clarify ERROR state meaning
  • KIP-671: Introduce Kafka Streams Specific Uncaught Exception Handler 
  • KIP-680: TopologyTestDriver should not require a Properties argument
  • KIP-689: Extend `StreamJoined` to allow more store configs
• Kafka 2.7:
  • KIP-450: Sliding Window Aggregations in the DSL
  • KIP-478 - Strongly typed Processor API
  • KIP-572: Improve timeouts and retries in Kafka Streams (partially implemented; completed in v2.8)
  • KIP-607: Add Metrics to Kafka Streams to Report Properties of RocksDB
  • KIP-613: Add end-to-end latency metrics to Streams (completed; partially implemented in v2.6)
  • KIP-616: Rename implicit Serdes instances in kafka-streams-scala
  • KIP-617: Allow Kafka Streams State Stores to be iterated backwards
  • KIP-626: Rename StreamsConfig config variable name
  • KIP-648: Renaming getter method for Interactive Queries
  • KIP-662: Throw Exception when Source Topics of a Streams App are Deleted
• Kafka 2.6:
  • KIP-221: Enhance DSL with Connecting Topic Creation and Repartition Hint
  • KIP-401: TransformerSupplier/ProcessorSupplier StateStore connecting
  • KIP-441: Smooth Scaling Out for Kafka Streams
  • KIP-444: Augment metrics for Kafka Streams (completed; partially implemented in v2.4 and v2.5)
  • KIP-446: Add changelog topic configuration to KTable suppress
  • KIP-447: Producer scalability for exactly once semantics (Kafka Streams changes; broker/client side changes already implemented v2.5)
  • KIP-557: Add emit on change support for Kafka Streams (partially implemented; reverted again in 2.8.0, 2.7.1, and 2.6.2 due to potential data loss)
  • KIP-571: Add option to force remove members in StreamsResetter
  • KIP-594: Expose output topic names from TopologyTestDriver
  • KIP-613: Add end-to-end latency metrics to Streams (partially implemented; completed in v2.7)
• Kafka 2.5: 
  • KIP-150 - Kafka-Streams Cogroup
  • KIP-444: Augment metrics for Kafka Streams (partially implemented in v2.4 and v2.5; completed in v2.6)
  • KIP-523: Add KStream#toTable to the Streams DSL
  • KIP-527: Add VoidSerde to Serdes
  • KIP-530: Consider renaming 'UsePreviousTimeOnInvalidTimeStamp' class to 'UsePartitionTimeOnInvalidTimeStamp'
  • KIP-535: Allow state stores to serve stale reads during rebalance
  • KIP-562: Allow fetching a key from a single partition rather than iterating over all the stores on an instance
• Kafka 2.4:
  • KIP-213 Support non-key joining in KTable
  • KIP-307: Allow to define custom processor names with KStreams DSL (completed; partially implemented in v2.3)
  • KIP-345: Introduce static membership protocol to reduce consumer rebalances (completed; partially implemented in v2.3)
  • KIP-429: Kafka Consumer Incremental Rebalance Protocol
  • KIP-444: Augment metrics for Kafka Streams (partially implemented in v2.4 and v2.5; completed in v2.6)
  • KIP-470: TopologyTestDriver test input and output usability improvements
  • KIP-471: Expose RocksDB Metrics in Kafka Streams (partially implemented; completion blocked)
  • KIP-474: To deprecate WindowStore#put(key, value)
  • KIP-479: Add StreamJoined config object to Join
  • KIP-528: Deprecate PartitionGrouper configuration and interface
• Kafka 2.3: 
  • KIP-258: Allow to Store Record Timestamps in RocksDB (partially implemented; not completed yet → inactive)
  • KIP-307: Allow to define custom processor names with KStreams DSL (partially implemented; completed in v2.4)
  • KIP-313: Add KStream.flatTransform and KStream.flatTransformValues (completed; partially implemented in v2.2)
  • KIP-331 Add default implementation to close() and configure() for Serializer, Deserializer and Serde
  • KIP-345: Introduce static membership protocol to reduce consumer rebalances (partially implemented; completed in v2.4)
  • KIP-428: Add in-memory window store
  • KIP-442: Return to default max poll interval in Streams
  • KIP-443: Return to default segment.ms and segment.index.bytes in Streams repartition topics
  • KIP-445: In-memory Session Store
  • KIP-453: Add close() method to RocksDBConfigSetter
  • KIP-462: Use local thread id for KStreams
• Kafka 2.2:
  

  • KIP-313: Add KStream.flatTransform and KStream.flatTransformValues (partially implemented; completed in v2.3)

  • KIP-376: Implement AutoClosable on appropriate classes that want to be used in a try-with-resource statement

  • KIP-386: Standardize on Min/Avg/Max metrics' default value

  • KIP-393: Time windowed serde to properly deserialize changelog input topic

  • KIP-414: Expose Embedded ClientIds in Kafka Streams

  • KIP-420: Add Single Value Fetch in Session Stores

...