...
Currently, the "data transfer" logic is locked within a single function, thus Kafka Connect API is inflexible for making such modifications. This is more so when a Kafka Connect API is provided via third party where the user cannot modify the source. Custom modifications from user to user would require modifying the source of Connect API.
Public Interfaces
Briefly list any new interfaces that will be introduced as part of this proposal or any existing interfaces that will be removed or changed. The purpose of this section is to concisely call out the public contract that will come along with this feature.
A public interface is any change to the following:
...
Binary log format
...
The network protocol and api behavior
...
Any class in the public packages under clientsConfiguration, especially client configuration
org/apache/kafka/common/serialization
org/apache/kafka/common
org/apache/kafka/common/errors
org/apache/kafka/clients/producer
org/apache/kafka/clients/consumer (eventually, once stable)
...
Monitoring
...
Command line tools and arguments
For the transformers, we require to introduce an abstract class which others can extend to create their own custom Transformer.
public abstract class Transformer<T1, T2> {
public abstract T2 transform(T1 t1);
public void initialize(Map<String, String> props) {}
}
Also, this would need a :
transformers=org.AbcTransformer,org2.XyzTransformer
property in the Kafka Connect config files to initialise the transformers, in the order specified in the above property.
...
Proposed Changes
Describe the new thing you want to do in appropriate detail. This may be fairly extensive and have large subsections of its own. Or it may be a few sentences. Use judgement based on the scope of the change.
...