This Confluence has been LDAP enabled, if you are an ASF Committer, please use your LDAP Credentials to login. Any problems file an INFRA jira ticket please.

Child pages
  • KIP-514: Add a bounded flush() API to Kafka Producer
Skip to end of metadata
Go to start of metadata


Status

Current stateUnder Discussion

Discussion thread: Here

JIRA: KAFKA-7711 

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Currently the call to Producer.flush() can be hang there for indeterminate time.

It would be great to add a bounded flush() API and timeout if producer is unable to flush all the batch records in a limited time. In this way the caller of flush() has a chance to decide what to do next instead of just wait forever.

Public Interfaces

A new API will be added to Kafka producer

KafkaProducer
@Override
public boolean flush(Duration timeout)

This API returns true if flush succeed in time, i.e all previously sent records have completed ; otherwise if times out before finish sending all records, this method returns false. 

Proposed Changes

Before this proposal, there is only one flush() API, which the caller will be blocked util all the pending batches(at the moment of calling flush()) succeed or exhaust all the retries.


After this proposal, the caller can specify an upper bound of the waiting time, and get notified wether flush succeed in this bounded time.

When timeout happens, this blocking call returns, but the sending/retry of incomplete batches does not get affected. 

Compatibility, Deprecation, and Migration Plan

There is not compatibility concern or migration plan needed for this change.

Rejected Alternatives

None.

  • No labels