Current state: Accepted
Discussion thread: https://lists.apache.org/thread/lxfq8y8co76mjwxhowwx6cjl0hw1mxpy
JsonConverter is used by Connect to convert raw JSON payloads, in bytes, to and from Connect schemas an values objects. When converting Connect objects to bytes, if the value is null and the field has a default value, JsonConverter always emits the default value. In some cases, for example when the value is explicitly set to null, it would be preferable to keep the null value instead.
In particular, if we use Debezium to stream changes out of a table, t1, in mysql:
and then insert one record into table
the record will store in MySQL as:
but the JsonConverter will always emit
It would be beneficial to be able to specify whether fields with a default value and set to null should be converted to null or the default value.
Add a configuration to JsonConverter:
Description: Whether to replace fields that have a default value and that are null to the default value. When set to true, the default value is used, otherwise null is used.
Update JsonConverter and JsonConverterConfig to handle the new configuration. It will affect serialization when the converter runs in a source pipeline and deserialization in sink pipelines.
Compatibility, Deprecation, and Migration Plan
The new configuration keeps the current behavior by default so it will not affect existing users. Users can optionally opt-in the new behavior by setting the configuration to false.
- Always take
nullfield which has default value. It will break the compatibility.
- Add the configuration to ConverterConfig