Current state: Under Discussion
Discussion thread: here
Sometimes, we found the users' functional interaction take a lot of time. At last, we figure out the root cause is that after we complete deploy or restart the servers.
The first message's delivery on each application server by kafka client will take much time.
After analyzing the source code about the first time's sending logic. The time cost is caused by the getting metadata before the sending. The latter's sending won't take the much time due to the cached metadata. The metadata's fetching logic is right and necessary. Thus, we still want to improve the experience for the first message's send/user first interaction.
So, This KIP try to raise one solution to improve it.
The solution is that we can provide one method for metadata fetch into Producer. When the application restarted/started, it can call it before mark the application is ready for handle requests. So, when the first request/record be handled. the metadata had been fetched so that it's handle's speed will be much faster.
note: waitOnMetadata(topic, null, time.milliseconds(), maxBlockTimeMs) is the existed method with provide modifier.
add new interface with tiny refactor which reduce the duplicated code.
Cluster getCluster(String topic, long maxBlockTimeMs);
The core code had been listed in Motivation part.
Compatibility, Deprecation, and Migration Plan
- What impact (if any) will there be on existing users?
no impact on existed users.
- If we are changing behavior how will we phase out the older behavior?
no changing older behavior.
- If we need special migration tools, describe them here.
- When will we remove the existing behavior?
no need to remove.
Compare the first record's send time cost to see if any improvement happens.
maybe we can provide one dedicated method with more naming instead of "getCluster".