Status

Current state: Accepted

Discussion thread: here 

JIRA: here 

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

window.size.ms and windowed.inner.serdes.class  are not true KafkaStreams configuration pieces, and are intended to be used for console client use cases. 

Public Interfaces

  • docs/streams/developer-guide/config-streams

In TimeWindowedSerializer:
  "windowed.inner.serializer.class"
In TimeWindowedDeserializer:
  "windowed.inner.deserializer.class"
   "window.size.ms"

In TimeWindowedSerializer:

   public static final String WINDOWED_INNER_SERIALIZER_CLASS = "windowed.inner.serializer.class";
    private static final String WINDOWED_INNER_SERIALIZER_CLASS = "Default serializer for the inner class of a windowed record. Must implement the " +
        "<code>org.apache.kafka.common.serialization.Serde</code> interface.";

In TimeWindowedDeserializer:

    /** {@code window.size.ms} */
    public static final String WINDOW_SIZE_MS_CONFIG = "window.size.ms";
    private static final String WINDOW_SIZE_MS_DOC = "Sets window size for the deserializer in order to calculate window end times.";

   public static final String WINDOWED_INNER_DESERIALIZER_CLASS = "windowed.inner.deserializer.class"";
    private static final String  WINDOWED_INNER_DESERIALIZER_CLASS  = "Default deserializer for the inner class of a windowed record. Must implement the " +
        "<code>org.apache.kafka.common.serialization.Serde</code> interface.";



In StreamsConfig.java:

  /** {@code window.size.ms
		* @deprecated since 3.8.0 Use "window.size.ms" in TimeWindowedDeserializer instead.} */
	@Deprecated
    public static final String WINDOW_SIZE_MS_CONFIG = "window.size.ms";
    private static final String WINDOW_SIZE_MS_DOC = "Sets window size for the deserializer in order to calculate window end times.";
...
   /** {@code windowed.value.serde.inner
 * @deprecated since 3.8.0 Use Use "windowed.inner.serializer.class" and "windowed.inner.deserializer.class" in TimeWindowedSerializer and TimeWindowedDeserializer instead. */
    @SuppressWarnings("WeakerAccess")
    @Deprecated
...
    public static final String WINDOWED_INNER_CLASS_SERDE = "windowed.inner.class.serde";
    private static final String WINDOWED_INNER_CLASS_SERDE_DOC = " Default serializer / deserializer for the inner class of a windowed record. Must implement the " +
        "<code>org.apache.kafka.common.serialization.Serde</code> interface. Note that setting this config in KafkaStreams application would result " +
        "in an error as it is meant to be used only from Plain consumer client.";


Proposed Changes

I propose we deprecate window.size.ms  in StreamsConfig as well as windowed.inner.serdes.class.

We will not define either config in the client's Config class, and instead opt to define a String variable with the config name in the relevant de/serializer class. There would be three new public String variables for three different configs:

In TimeWindowedSerializer:
  - define a constant for "windowed.inner.serializer.class"
In TimeWindowedDeserializer:
  - define a constant for "windowed.inner.deserializer.class"
  - define a constant for "window.size.ms"

We update the windowed de/serializer implementations to check for the new configs (ie "windowed.inner.de/serializer.class") and use the provided de/serializer class, if one was given. If the new configs are not present, they would fall back to the original/current logic (ie that based on the old "windowed.inner.serde.class" config). 

If both are specified, the new piece of config takes preference. 

Compatibility, Deprecation, and Migration Plan

  • Deprecating it in StreamsConfig  shouldn't impact current users negatively other than them seeing a warning when they use it in KStreams.  

  • We should change the config docs first to deprecate. In the future, moving the configuration to ClientConfig could be called for. See https://github.com/apache/kafka/pull/14360 for a PR in progress, as well as a discussion of this issue.
  • Post docs change and deprecation, for major release 5. Deprecation itself can be for next minor release. 

Test Plan

Could be covered with a unit test to check that a warning is displayed when used.

 Also have a unit test to make sure the new "windowed.inner.de/serializer.class" takes preference in the case that both it, and the old "windowed.inner.serde.class" config are both specified

Rejected Alternatives

Currently N/A.

  • No labels