Status
Current state: Under discussion
Discussion thread: https://lists.apache.org/thread.html/a5ee2d569448dd57647dca93f288405c13785dec9cb4a0cbddfedd35@%3Cdev.kafka.apache.org%3E
JIRA: KAFKA-7658 - Getting issue details... STATUS
Motivation
For users who cannot read their source topic as a changelog stream from the beginning we need to provide a way for event streams to be translated into changelog stream. As pointed out by Guozhang Wang this should not be confused with KStream.reduce(), these functions should:
- completely change the semantics of the input stream from event stream to changelog stream any null-values will still be serialized, and if the resulted bytes are also null they will be interpreted as "deletes" to the materialized KTable (i.e. tombstones in the changelog stream).
- materializing these KTables should only be allowed if the overloaded function with Materialized is used (and if optimization is turned on it may still be only logically materialized if the queryable name is not set).
Public Interfaces
Two functions will need to be added to the KStream interface:
<VR> KTable<K,VR> toTable() <VR> KTable<K,VR> toTable(final Materialized<K, VR, KeyValueStore<Bytes, byte[]>> materialized)
Proposed Changes
Adding two new functions to the Kstreams DSL.
Compatibility, Deprecation, and Migration Plan
No users will be impacted as this is just addition of two new methods.
Rejected Alternatives
None.