Motivation

The ConfigDef.define() methods, which create ConfigKey instances,  have increased the number of parameters past the point where a Builder is recommended.  The ConfigKey class is not final but extending it has become difficult.

Developing additional functionality for ConfigKey requires changes to the current ConfigKey and ConfigDef classes.  Implementing a ConfigKey.Builder class will make development of new ConfigKey functionality easier because changes can be more localized.

Public interfaces

Adds a new interface to construct the ConfigKey that may be used in parallel with the existing ConfigDef.define() methods.

Proposed Changes

This proposal adds a ConfigKey.Builder class that will construct ConfigKey instances equivalent to the ConfigDef.define() methods while allowing for extensibility.   This proposal does not remove any define() method nor mark them as deprecated.

The builder must:

• Be extensible since ConfigKey is.
• Be created by calling a static ConfigKey.builder(name), where name is the name of the config parameter.
• Provide reasonable defaults as found in the current ConfigDef.define() methods.
• during its build() method call a new constructor in ConfigKey that takes the Builder as an argument.

Compatibility, Deprecation, and Migration Plan

The result of the build() method is equivalent to the ConfigKey.defin() method, as such no compatibility issues are expected.

While ConfigDef.define() could be deprecated, there are no plans to do so under this proposal.  If accepted, it is expected that the builder will be used in favor of the ConfigDef.define() method in future development due to ease of use.  Deprecation of ConfigDef.define() is left to a future KIP.

No migration is proposed.  Developers may utilize either the current ConfigDef.define() methods or the ConfigKey.builder() methods as they see fit.

Test Plan

Unit tests to verify that builder constructed ConfigKeys are equivalent to current ConfigKeys,define() method results.

Unit tests to verify that the builder can be extended.

Rejected Alternatives

none

Discussion: https://lists.apache.org/thread/lyqx89cyg6lonk9ncfrnmql5v0sp4n9y