Status

Current state: Under Discussion

...

Motivation

Currently, IQ throws InvalidStateStoreException for any types of error, that means a user cannot handle different types of error.

Because of that, we should throw different exceptions for each type.

Proposed Changes

The To distinguish different types of error, we need to handle all InvalidStateStoreExceptions better during these public methods invoked. The main change is to introduce new exceptions that extend from InvalidStateStoreException. InvalidStateStoreException is not thrown at all anymore, but only new sub-classes.

# ThrowTwo tocategory user exceptionexceptions
public class StateStoreMigratedExceptionRetryableStateStoreException extends InvalidStateStoreException
public class StateStoreRetryableExceptionFatalStateStoreException extends InvalidStateStoreException


# Retryable exceptions
public class StateStoreFailExceptionStreamThreadNotStartedException extends InvalidStateStoreException

 
# Internal exception
RetryableStateStoreException
public class StateStoreClosedExceptionStreamThreadRebalancingException extends InvalidStateStoreExceptionRetryableStateStoreException
public class StateStoreEmptyExceptionStateStoreMigratedException extends InvalidStateStoreException RetryableStateStoreException


# Fatal exceptions
public class StreamThreadNotRunningException extends InvalidStateStoreException

Three categories exception throw to the user:

FatalStateStoreException

Various state store exceptions can classify into two category exceptions: RetryableStateStoreException and FatalStateStoreException. The user can use the two exceptions if they only need to distinguish whether it can retry.

• Retryable exceptions
  • StreamThreadNotStartedException: will be thrown when streams thread state is CREATED, the user can retry until to RUNNING.
  • StreamThreadRebalancingException: will be thrown when stream thread is not running and stream state is REBALANCING, the user just
  StateStoreRetryableException: The application instance in the state of rebalancing, the user just need
  • retry and wait until rebalance finished (RUNNING).
  • StateStoreMigratedException:
  The store got migrated and not hosted in application instance, the users need to rediscover the store.
• StateStoreFailException: Fatal error when access state store, the user cannot retry or rediscover.

Three internal exceptions: StateStoreClosedException, StateStoreEmptyException, StreamThreadNotRunningException

The internal exception will be wrapped as user's category exception finally. The purpose of the internal exceptions is to distinguish between the different kinds of InvalidStateStoreException. For example, StreamThreadStateStoreProvider#stores() will throw StreamThreadNotRunningException(internal exception). And then the internal exception will be wrapped as StateStoreRetryableException or StateStoreFailException during the KafkaStreams.store() and throw to the user.

• • will be thrown when state store already closed and stream state is REBALANCING.
• Fatal exceptions
  • KafkaStreamsNotRunningException: will be thrown when stream thread is not running and stream state is PENDING_SHUTDOWN / NOT_RUNNING / ERROR. The user cannot retry when this exception is thrown.
  • StateStoreNotAvailableException: will be thrown when state store closed and stream thread is PENDING_SHUTDOWN / NOT_RUNNING / ERROR. The user cannot retry when this exception is thrown.
  • UnknownStateStoreException: will be thrown when passing an unknown state store.

The following is the public methods that users will call to get state store instance:

• KafkaStreams
  
  • store(storeName, queryableStoreType)

...

The following is the public method methods that users will call to get store values:

• KafkaStreams
  
  • store()
• interface ReadOnlyKeyValueStore(class CompositeReadOnlyKeyValueStore)
  • get(kkey)
  • range(from, to)
  • all()
  • approximateNumEntries()
• ReadOnlySessionStoreinterface ReadOnlySessionStore(CompositeReadOnlySessionStoreclass CompositeReadOnlySessionStore)
  
  • fetch(kkey)
  • fetch(from, to)
  ReadOnlyWindowStore(CompositeReadOnlyWindowStore
• interface ReadOnlyWindowStore(class CompositeReadOnlyWindowStore)
  
  • fetch(key, time)

  • fetch(

    k

    key,

    rf

    from,

    tt

    to)

  • fetch(from, to, rffromTime, tttoTime)
  • all()
  • fetchAll()
• KeyValueIterator(DelegatingPeekingKeyValueIterator)
  
  • next()
  • hasNext()
  • peekNextKey()
• WindowStoreIterator(MeteredWindowStoreIterator)
  
  • next()
  • hasNext()
  • peekNextKey()

...

...

• • (

...

• • from,

...

• • to)
  • @Deprecated fetch(key, timeFrom, timeTo)
  • @Deprecated fetch(from, to, timeFrom, timeTo)
  • @Deprecated fetchAll(timeFrom, timeTo)
• interface KeyValueIterator(class DelegatingPeekingKeyValueIterator)
  
  • next()
  • hasNext()
  • peekNextKey()

During user call one of above methods, we should check KafkaStreams state by the following rule when InvalidStateStoreException is thrown:

• If state is RUNNING or REBALANCING:
  • StateStoreClosedException: should be wrapped to StateStoreRetryableException
  • StreamThreadNotRunningException: should be wrapped to StateStoreRetryableException
  • StateStoreEmptyException: should be wrapped to StateStoreMigratedException
• if state is PENDING_SHUTDOWN or ERROR or NOT_RUNNING:
  • wrap InvalidStateStoreException(include sub classes) to StateStoreFailException

Call Trace

...

• CompositeReadOnlySessionStore#fetch(key) (v)
  • WrappingStoreProvider#stores() (v)
    
    • StreamThreadStateStoreProvider#stores() (v)
  • MeteredSessionStore#fetch(key)
    • MeteredSessionStore#findSessions()
      • CachingSessionStore#findSessions()
        • AbstractStateStore#validateStoreOpen() (v)
        • ChangeLoggingSessionBytesStore#findSessions()
          • RocksDBSessionStore.findSessions(k)
            • RocksDBSessionStore.findSessions(from, to)
              • RocksDBSegmentedBytesStore#fetch()
                • SessionKeySchema#segmentsToSearch()
                  • Segments#segments() (v)
                    • Segments#getSegment()
                      • ConcurrentHashMap#get()
                      • Segments#isSegment()
                      • return segment
                    • retiurn segments
                  • return
                • return new SegmentIterator()
              • return new WrappedSessionStoreIterator()
            • return
          • return
        • return new MergedSortedCacheSessionStoreIterator()
      • return new MeteredWindowedKeyValueIterator()
    • return
  • MeteredWindowedKeyValueIterator#hasNext()
    • MergedSortedCacheSessionStoreIterator#hasNext()
      AbstraceMergedSortedCacheStoreIterator#hasNext()
      • FilteredCacheIterator#hasNext()
      • WrappedSessionStoreIterator#hasNext()
        • SegmentIterator#hasNext()
          • Segment.range(from, to)
            RocksDBStore.range(from, to)
            • RocksDBStore.validateStoreOpen() (v)
            • return new RocksDBRangeIterator()
          • return
        • return
      • return
    • return iterator(MeteredWindowedKeyValueIterator)
• MeteredWindowedKeyValueIterator#next()
  • MergedSortedCacheSessionStoreIterator#next()
    AbstractMergedSortedCacheStoreIterator#next()
    • AbstractMergedSortedCacheStoreIterator#hasNext()
    • FilteredCacheIterator#hasNext()
    • WrappedSessionStoreIterator#hasNext()
      • SegmentIterator#hasNext()
        • Segment.range(from, to)
          RocksDBStore.range(from, to)
          • RocksDBStore.validateStoreOpen() (v)
          • return new RocksDBRangeIterator()
        • return
      • return
    • MergedSortedCacheSessionStoreIterator#nextStoreValue()
      AbstractMergedSortedCacheStoreIterator#nextStoreValue()
      
      • WrappedSessionStoreIterator#next()
        • SegmentIterator#next()
          • RocksDBRangeIterator#next()
            RocksDbIterator#next()
            • RocksDbIterator#getKeyValue()
            • RocksIterator#next()
            • return entry
          • return
        • return
      • return
    • return
  • return

...

Compatibility, Deprecation, and Migration Plan

...

Rejected Alternatives

None.