Status

Current state: Under Discussion

Discussion thread: here

JIRA:  KAFKA-15597 - Getting issue details... STATUS

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

In many use cases it is required to drop a few specific Kafka headers but a set of headers whose names can also dynamically change (e.g. when used with some end-to-end-encryption libraries, tracing etc.). To prevent those headers following a special pattern (which for example may or may not comply with downstream system format) to be further forwarded/processed downstream (e.g.header forwarding in HTTP Sinks), this KIP proposes to add a wildcard/regexp matching feature to the DropHeaders Transform.

Public Interfaces

It is proposed to add new configuration is to the SMT org.apache.kafka.connect.transforms.DropHeaders to allow regexp-based matching of headers to be dropped:

Name: headers.patterns

Description:List of regular expressions to match of the headers to be removed.

Type: String (List)

Default: "$^" (match empty)

Proposed Changes

It is proposed to use regexp-matchers inside the apply method in addition to a set-based contains (existing implementation).

Compatibility, Deprecation, and Migration Plan

If the functionality is added with regexp-matching in addition to the existing implementation, no breaking will be done, no migration is required as the additional feature comes with a standalone configuration option

Rejected Alternatives

None.