This page is meant as a template for writing a KIP. To create a KIP choose Tools->Copy on this page and modify with your content and replace the heading with the next KIP number and a description of your issue. Replace anything in italics with your own description.
Current state: Accepted
Discussion thread: here
JIRA: KAFKA-12379
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
To provide offset translation, MirrorMaker2 uses the mm2-offset-syncs topic. This topic is populated by the Source connector and then used by the Checkpoint connector to convert consumer group offsets. This topic is created on the source cluster. It means that when mirroring records from a source cluster to a target cluster, MirrorMaker2 requires write permission on the source cluster (at least for mm2-offset-syncs).
In secured environments, it would be preferrable to only grant reader access to the source cluster to MirrorMaker2.
As this topic is internal to MirrorMaker2 and never exposed to users we could make its location configurable by administrators. This would allow moving that topic to the target cluster if needed and only grant reader access to the source cluster.
The proposal is to introduce a new setting to MirrorMaker2 to control the location of the offset syncs topic.
Name: offset-syncs.topic.location
Doc: The location of the offset-syncs topic.
Type: String
Default: source
Valid values: source, target
Importance: Low
In dedicated mode, it can be used like:
clusters = us-east, us-west us-east->us-west.enabled true us-east->us-west.offset-syncs.topic.location = target |
In Connect distributed mode, this setting needs to be set on both the Source and Checkpoint connectors.
Based on the new setting, the Source and Checkpoint connectors will use either the source cluster (default) or the target cluster for offset syncs.
The default value preserves the current behaviour so there is no impact on existing users.
Existing users that want to move the offset-syncs topic will need to stop their connectors, clear the offset-syncs topic and restart the connectors with the new configuration.
None