Current state: Accepted
Discussion thread: here
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
In Kafka Streams the
KStream interface has several
transformValues methods, where as the
KTable interface does not. The related
mapValues method is available on both
transformValue method is more flexible than
- Stateful implementations.
transformValuesaccepts a supplier of transformers, rather than the mapper instance
mapValuesaccepts. The supplier is used to instantiate a transformer per stream task, meaning the implementation does not need to be thread-safe. Conversely,
mapValuesshares the passed in mapper instance across stream tasks, so the implementations must be thread-safe.
- State-store access:
transformValuesallows access to existing state-stores, where as
- Richer API: the
ValueTransformerinterface is much richer than the
ValueMapper, supporting both
There is no conceptual reason not to support the richer
transformValues on the
The following methods would be added to the Java
The following methods would be added to the Scala KTable class.
The new methods on `KTableImpl` will add a new
KTableTransformValues processor node and attach any state stores. The new
KTableTransformValues will be implemented in a similar manner to other processors, instantiating the user supplied transformer once per task.
The `ProcessorContext` passed to the `init()` method of transformer implementations will be restricted: any call to any variant of the `forward()` method will throw a `StreamsException`. This will stop implementations outputting values with a new key.
Compatibility, Deprecation, and Migration Plan
Users must upgrade to new version if they want to use this functionality.
Unit tests to cover new classes and methods. Integration or system test are not required.
Include overloads that take a `ValueTransformerSupplier`, matching the overloads available on `KStream`. These were no included as it keeps the interface more succinct, users can ignore the key value if they do not need it, and likely these overloads on the `KStream` interface will be deprecated, in favour of the 'WithKey' variants, in time.