Child pages
  • KIP-147: Add missing type parameters to StateStoreSupplier factories and KGroupedStream/Table methods
Skip to end of metadata
Go to start of metadata

Status

Current state"Under Discussion"

Discussion thread: here

JIRA: TBD

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

The recommended practice to create a StateStoreSupplier is as per the following example:

However, StateStoreSupplier is a generic interface that takes the StateStore type as a parameter:

In the above example that type parameter is lost as the build() method returns a raw type.

As StateStoreSupplier is passed to count/reduce/aggregate etc. methods on KGroupedStream or KGroupedTable, the compiler cannot detect if a supplier for the wrong kind of store is provided.

The other parameters to those methods, such as Serdes, Reducers, etc are type-parameterised by the key and value types allowing compile-time type checks.

The StateStoreSupplier argument stands out as a raw type. Making it type-parameterised will help detect at compile time errors such as when someone refactors their app to use a different typo of aggregations (e.g. TimeWindowed vs SessionWindowed) and forgets to change the StateStoreSupplier passed in.

Public Interfaces

  • KGroupedStream

    • KTable<K, Long> count(final StateStoreSupplier<KeyValueStore<K, Long>> storeSupplier);

    • <W extends Window> KTable<Windowed<K>, Long> count(final Windows<W> windows,
                                                                                                     final StateStoreSupplier<WindowStore<K, Long>> storeSupplier);

    • KTable<Windowed<K>, Long> count(final SessionWindows sessionWindows,
                                                                  final StateStoreSupplier<SessionStore<K, Long>> storeSupplier);

    • KTable<K, V> reduce(final Reducer<V> reducer,
                                         final StateStoreSupplier<KeyValueStore<K ,V>> storeSupplier);

    • <W extends Window> KTable<Windowed<K>, V> reduce(final Reducer<V> reducer,
                                                                                                  final Windows<W> windows,
                                                                                                  final StateStoreSupplier<WindowStore<K, V>> storeSupplier);

    • KTable<Windowed<K>, V> reduce(final Reducer<V> reducer,
                                                              final SessionWindows sessionWindows,

                                                              final StateStoreSupplier<SessionStore<K, V>> storeSupplier);

    • <VR> KTable<K, VR> aggregate(final Initializer<VR> initializer,
                                                           final Aggregator<? super K, ? super V, VR> aggregator,

                                                           final StateStoreSupplier<KeyValueStore<K, VR>> storeSupplier);

    • <W extends Window, VR> KTable<Windowed<K>, VR> aggregate(final Initializer<VR> initializer,
                                                                                                                 final Aggregator<? super K, ? super V, VR> aggregator,
                                                                                                                 final Windows<W> windows,
                                                                                                                 final StateStoreSupplier<WindowStore<K, VR>> storeSupplier);

    • <T> KTable<Windowed<K>, T> aggregate(final Initializer<T> initializer,

                                                                          final Aggregator<? super K, ? super V, T> aggregator,

                                                                          final Merger<? super K, T> sessionMerger,

                                                                          final SessionWindows sessionWindows,

                                                                          final Serde<T> aggValueSerde,

                                                                          final StateStoreSupplier<SessionStore<K, T>> storeSupplier);

  • KGroupedTable
    • KTable<K, Long> count(final StateStoreSupplier<KeyValueStore<K, Long>> storeSupplier);

       

    • KTable<K, V> reduce(final Reducer<V> adder,
                                         final Reducer<V> subtractor,
                                         final StateStoreSupplier<KeyValueStore<K, V>> storeSupplier);

       

    • <VR> KTable<K, VR> aggregate(final Initializer<VR> initializer,
                                                           final Aggregator<? super K, ? super V, VR> adder,
                                                           final Aggregator<? super K, ? super V, VR> subtractor,
                                                           final StateStoreSupplier<KeyValueStore<K, VR>> storeSupplier);

  • Stores
    • 2 new public interfaces:
      • changes to PersistentKeyValueFactory :
        • PersistentWindowFactory<K, V> windowed(final long windowSize, long retentionPeriod, int numSegments, boolean retainDuplicates);

        • PersistentSessionFactory<K, V> sessionWindowed(final long retentionPeriod);

        • StateStoreSupplier<KeyValueStore<K, V>> build();

      • changes to InMemoryKeyValueFactory:
        • StateStoreSupplier<KeyValueStore<K, V>> build();

 

Proposed Changes

The new usage would be e.g.:

 

Compatibility, Deprecation, and Migration Plan

 

These changes are, in general, not backward compatible in that some mechanical but manual changes to user code may be required.

Most case including usage as per the documentation (see example in Motivation) no changes would be required, although users should be encouraged to add type parameters to the previously used raw types to get rid of compilation warnings or to remove @SupressWarnings annotations.

However, some mechanical changes would be required if StateStoreSupplier type parameterised with the raw SessionStore type was used in a cast, for instance. See: KGroupedStreamImplTest.java#L98

Another case where changes are needed would be if results of PersistentKeyValueFactory .sessionWindowed() or .windowed() or results of calls to enableLogging(), disableLogging(), enableCaching() on return values of .sessionWindowed() or .windowed() were assigned to a variable, field or used as a parameter. In that case the type of the variable, field or parameter would need changing from PersistentKeyValueFactory to PersistentSessionFactory<K, V> or PersistentWindowFactory<K, V> respectively.

To make migration straightforward upgrade note shall be provided in the documentation.

Test Plan

Only re-run of existing tests is envisaged at this time.

Rejected Alternatives

Node considered.

  • No labels