...
Introduce a new configuration to the JsonConverter named decimal.format to control whether source converters will serialize decimals in numeric or binary formats. The value will be case insensitive and can be either "BASE64" (default, to maintain compatibility) or "NUMERIC".
Proposed Changes
There are four proposed We propose the following changes:
- Define a new decimal.format configuration property on JsonConverter to specify the serialization format for Connect DECIMAL logical type values with two allowed literals as the configuration property:
- (default) BASE64 specifies the existing behavior of serializing DECIMAL logical types as base64 encoded binary data (e.g. "D3J5" in the example above); and
- NUMERIC will serialize Connect DECIMAL logical type values in JSON as a number representing that decimal (e.g. 10.2345 in the example above)
- The JsonConverter deserialization method currently expects only a BinaryNode, but will be changed to also handle NumericNode by calling NumericNode.decimalValue().
- JsonDeserializer will now default floating point deserialization to BigDecimal to avoid losing precision. This may impact performance when deserializing doubles - a JMH microbenchmark on my local MBP, this estimated about 3x degradation for deserializing JSON floating points. If the connect schema is not the decimal logical type, the JsonConverter will convert this BigDecimal value into the corresponding floating point java object.
- Configure the JsonConverter for internal topics with `decimal.format=NUMERIC` so that if the DECIMAL types will be serialized in a more natural representation. This is safe since connect internal topics do not currently use any decimal types.
To understand behavior of this configuration with and without schemas, refer to the table below:
| JsonConverter | |||
|---|---|---|---|
| Schemas Enabled | Schemas Disabled | JsonDeserializer | |
| DECIMAL (BASE64) | returns DECIMAL logical type | throws DataException | stores BinaryNode (byte[] data) |
| DECIMAL (NUMERIC) | returns DECIMAL logical type | returns float64 (lossy) | stores NumericNode (BigDecimal data) |
| FLOAT64 (Legacy) | returns FLOAT64 | returns float64 | stores NumericNode (Double data) |
| FLOAT64 (KIP-481) | returns FLOAT64 | returns float64 | stores NumericNode (BigDecimal data) |
Compatibility, Deprecation, and Migration Plan
...