Current state: Under Discussion

Discussion thread: here

JIRA: here

PR: here


With DefaultReplicationPolicy, the heartbeats topic name is hard-coded. Instead, this should be configurable, so users can avoid collisions with the "heartbeats" topics of other systems.

Public Interfaces

New MirrorConnectorConfig:

default.replication.policy.heartbeats.topic.nameStringheartbeatsName of the heartbeats topic

Since it is not a significant change, I implemented it in advance. Draft PR:


Compatibility, Deprecation, and Migration Plan

  • Since this feature does not change the default topic name, existing users won't be affected, unless they change the property intentionally.
  • If we change the behavior in flight (e.g.: start the MM2 with default: heartbeats topic, stop MM2, then start it with a new custom-heartbeats topic) then the original topic becomes stale/dangling and the MM2 will use the new topic to track the heartbeats

Test Plan

I tested the change manually on 2 local clusters.
I wrote a couple of unit test and also wrote an integration test. However, I am not included the integration test in the PR because it seemed overkill for this small change, I don't want to introduce extra heavyweight tests, to slow down pipeline without a good reason (but, I am happy to include it if you think it is necessary)

Documentation Plan

We should introduce the new property in the 3.8.1 MirrorMaker Common Configs section

Rejected Alternatives

  • Divert from the DefaultReplicationPolicy and use a custom one.

This is a feasible solution proposed by Chris Egerton KIP-690, however, I think that it would be beneficial for the users to be able to configure the heartbeat topic name with the DefaultReplicationPolicy in an easy, no code way.

  • No labels