Current state: Discarded, and superseded by KIP-215: Add topic regex support for Connect sinks
Discussion thread: here
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
For now we provide topic names in a properties file as following:
If there are a few topics to deal with, It's ok to list all the topics in a comma-seperated way. But when you have to deal with hundreds or thousands of topics, it doesn't help. In those situations, we would like to use a wildcard expression to specify topics.
client configuration file
- add an aditional option "topics.regex"
- a user can only specify one of "topics" or "topics.regex"
- if a user specifies "topics" in their config file, it works the same way as the current version.
- if a user specifies "topics.regex", it can handle all the topics matching the regular expressions.
Compatibility, Deprecation, and Migration Plan
- What impact (if any) will there be on existing users?
- If we are changing behavior how will we phase out the older behavior?
- If we need special migration tools, describe them here.
- Not needed
- When will we remove the existing behavior?
It is possible to update the interpretation of the existing option "topics" in a way that our users can specify reglar expressions there. But since there are some special characters in the regular expression that can be used in topic names, eg. "." and "-", we need some workaround. Also this aproach has some effects on existing users.