This Confluence has been LDAP enabled, if you are an ASF Committer, please use your LDAP Credentials to login. Any problems file an INFRA jira ticket please.

Child pages
  • KIP-279: Fix log divergence between leader and follower after fast leader fail over

Versions Compared


  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents


Current state"Under Discussion"Accepted

Discussion threadhere


serverASF JIRA


Now what happens next depends on the last offset of the batch appended in epoch 2. On becoming follower, broker B will send an OffsetForLeaderEpoch request to broker A with epoch 2. Broker A will respond with offset 11 21 (the start offset of the first leader epoch larger than 2, since broker A does not know about epoch 2).  There are three cases:


  1. In the current protocol, when the follower sends OffsetForLeaderEpoch request for the partition to the leader, the request includes the latest Leader Epoch in the follower's Leader Epoch Sequence. This step and steps before that are the same as described in KIP-101. 
  2. The leader responds with the largest epoch less than or equal to the requested epoch (LeaderEpoch) and the end offset of this epoch (LastOffset).
  3. If the follower has LeaderEpoch received from the leader in its Leader Epoch Sequence file, then we go to step 4. Otherwise, 
    1. The follower truncates to the end offset of the largest epoch less than LeaderEpoch, and
    2. The follower sends OffsetForLeaderEpoch request with the largest epoch less than LeaderEpoch, and
    1.  Steps 2 and 3 repeat until the follower receives the epoch it knows about (the epoch is in the follower's Leader Epoch Sequence file).
  4. The By following steps 2 and 3, the follower truncates all offsets with epochs larger than the epoch received from the leader (LeaderEpoch), and then . In this step, the follower truncates its log to the leader's LastOffset, if leader's LastOffset is smaller than follower's Log End Offset.
  5. The follower starts fetching from the leader and the remainder of the protocol remains unchanged.