Versions Compared

Old Version 118

changes.mady.by.user David Jacot

Saved on

compared with

New Version 119

changes.mady.by.user David Jacot

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Note this is a joint work proposed by David Jacot, Guozhang Wang and Jason Gustafson.

Info
  • The consumer rebalance protocol with server side assignors is GA in Apache Kafka 4.0.
  • The Streams related parts are obsolete and have been superseded by KIP-1071: Streams Rebalance Protocol.
  • The client side assignors are not implemented yet.
  • The topic ids for offset fetch/commit are planned for 4.1/4.2.

Motivation

It has been about 8 years since we introduced the so-called new consumer which does group membership and rebalancing through Kafka. Although it was a huge improvement over the old Zookeeper based consumer, it has still been a major pain point from an operation perspective. There are multiple reasons for this, let’s dive into them:

...