Current state: Under Discussion
Discussion thread: Here
Currently the call to Producer.flush() can be hang there for indeterminate time.
It would be great to add a bounded flush() API and timeout if producer is unable to flush all the batch records in a limited time. In this way the caller of flush() has a chance to decide what to do next instead of just wait forever.
A new API will be added to Kafka producer
This API returns normally if flush succeed in time, i.e all previously sent records have completed ; otherwise if times out before finish sending all records, an TimeoutException will be thrown.
Before this proposal, there is only one flush() API, which the caller will be blocked util all the pending batches(at the moment of calling flush()) succeed or exhaust all the retries.
After this proposal, the caller can specify an upper bound of the waiting time, and get notified wether flush succeed in this bounded time.
When timeout happens, this blocking call returns, but the sending/retry of incomplete batches does not get affected.
Compatibility, Deprecation, and Migration Plan
There is not compatibility concern or migration plan needed for this change.