Check out the docs at Apache Kafka web site: http://kafka.apache.org/documentation/streams/
Sub-pages:
- Using Kafka Streams:
- Development:
- Internals:
Find more links about Kafka Streams at Kafka Ecosystem page.
Kafka Streams related KIPs:
Below is a list of KIPs that are not release yet. Go to Kafka Streams KIP Overview for KIPs by release (including discarded KIPs).
Under discussion
- KIP-418: A method-chaining way to branch KStream
- KIP-424: Allow suppression of intermediate events based on wall clock time
- KIP-439: Cleanup built-in Store interfaces
- KIP-448: Add State Stores Unit Test Support to Kafka Streams Test Utils
- KIP-508: Make Suppression State Queriable
- KIP-513: Distinguish between Key and Value serdes in scala wrapper library for kafka streams
- KIP-540: Implement per key stream time tracking
- KIP-560: Auto infer external topic partitions in stream reset tool
- KIP-572: Improve timeouts and retries in Kafka Streams
- KIP-591: Add Kafka Streams config to set default store type
- KIP-598: Augment TopologyDescription with store and source / sink serde information
- KIP-614: Add Prefix Scan support for State Stores
Adopted
- inactive, not (fully) implemented (feel free to pick up):
- KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner (partially implemented in v1.1)
- KIP-258: Allow to Store Record Timestamps in RocksDB (partially implemented in v2.3)
- KIP-300: Add Windowed KTable API in StreamsBuilder (accepted but postponed because current code base does not allow to implement it as proposed)
- KIP-328: Ability to suppress updates for KTables (partially implemented in v2.1)
- KIP-471: Expose RocksDB Metrics in Kafka Streams (partially implemented in v2.4; completion deferred to next major release)
- WIP (open PR / in review):
- KIP-216: IQ should throw different exceptions for different errors
- KIP-399: Extend ProductionExceptionHandler to cover serialization exceptions
- KIP-466: Add support for List<T> serialization and deserialization
- KIP-478 - Strongly typed Processor API
- KIP-557: Add emit on change support for Kafka Streams (partially implemented in v2.6)
- KIP-571: Add option to force remove members in StreamsResetter
- KIP-607: Add Metrics to Kafka Streams to Record the Memory Used by RocksDB
- KIP-613: Add end-to-end latency metrics to Streams
- upcoming release Kafka 2.6 (merged):
- KIP-221: Enhance DSL with Connecting Topic Creation and Repartition Hint
- KIP-401: TransformerSupplier/ProcessorSupplier StateStore connecting
- KIP-441: Smooth Scaling Out for Kafka Streams
- KIP-444: Augment metrics for Kafka Streams (completed; partially implemented in v2.4 and v2.5)
- KIP-446: Add changelog topic configuration to KTable suppress
- KIP-447: Producer scalability for exactly once semantics (Kafka Streams changes; broker/client side changes already implemented 2.5)
- KIP-557: Add emit on change support for Kafka Streams (partially implemented)
- KIP-594: Expose output topic names from TopologyTestDriver
Inactive (feel free to pick up and resume the discussion):
- KIP-95: Incremental Batch Processing for Kafka Streams
- KIP-159: Introducing Rich functions to Streams
- KIP-260: add primary join operation for Stream-Stream join (WIP) (draft)
- KIP-311: Async processing with dynamic scheduling in Kafka Streams
- KIP-314: KTable to GlobalKTable Bi-directional Join
- KIP-326: Schedulable KTable as Graph source
- KIP-335: Consider configurations for KafkaStreams
- KIP-362: Support dynamic gap session window
- KIP-378: Enable Dependency Injection for Kafka Streams handlers
- KIP-406: GlobalStreamThread should honor custom reset policy
- KIP-408: Add Asynchronous Processing To Kafka Streams
- KIP-450: Sliding Window Aggregations in the DSL
- KIP-457: Add DISCONNECTED status to Kafka Streams
- KIP-459: Improve KafkaStreams#close
- KIP-463: Auto-configure non-default Serdes passed alongside the TopologyBuilder
- KIP-XXX: [Streams] Support multiple topic resolution in TopicNameExtractor (draft)