Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Discussion thread: here

JIRA: KAFKA-3209

Released: 0.10.2.0

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Note

The framework for Single Message Transforms was released on 0.10.2.0 but only some of the built-in transformations were included with that version. The table below indicates what version each transformation was or will be released with. A few don't have the exact name as listed in the KIP because they were found to be slightly inaccurate during code review.

TransformationVersion
InsertField0.10.2.0 
ReplaceField0.10.2.0 
MaskField0.10.2.0 
ValueToKey0.10.2.0 
HoistField0.10.2.0 
ExtractField0.10.2.0 
SetSchemaMetadata0.10.2.0 
TimestampRouter0.10.2.0 
RegexRouter0.10.2.0
Flatten0.11.0.0
Cast0.11.0.0 
TimestampConverter0.11.0.0?

The Kafka documentation also includes references for each transformation.

 

Motivation

This proposal is for adding a record transformation API to Kafka Connect as well as certain bundled transformations. At the same time, we should not extend Connect's area of focus beyond moving data between Kafka and other systems. We will only support simple 1:{0,1} transformations – i.e. map and filter operations.

...