You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Next »


Status

Current state: Under Discussion

Discussion thread: here [Change the link from the KIP proposal email archive to your own email thread]

JIRA: KAFKA-4426

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Consumer API currently has one close() method without a timeout. KAFKA-3703 added graceful close of the consumer with a hard-coded timeout of 5 seconds to ensure that consumers complete pending commits and leave the group gracefully before closing connection to the coordinator.

Producer interface has two close() methods.

Produce close methods
public void close();
public void close(long timeout, TimeUnit unit);

The first variant waits for graceful close without a time limit (timeout=Long.MAX_VALUE). The second attempts to close gracefully, but terminates the produce, aborting any pending requests after the timeout.

For consistency with producers, consumers should also expose the same two variants for close.

Public Interfaces

A new close method with timeout will be added to the Consumer interface. The existing close() method will attempt to close gracefully (timeout=Long.MAX_VALUE).

Consumer close with timeout
/**
 * Tries to close the consumer cleanly within the specified timeout. If the close does not complete within the
 * timeout, force close the producer.
 */
public void close(long timeout, TimeUnit unit);

 

 

Proposed Changes

The existing close() method without a timeout will attempt to close the consumer gracefully with a timeout of Long.MAX_VALUE instead of the hard-coded 5 second timeout. The current close logic that attempts to commit offsets if the coordinator is known and leave group will remain the same but will be executed with the specified timeout instead of the hard-coded 5-second timeout.


Compatibility, Deprecation, and Migration Plan

What impact (if any) will there be on existing users?

Versions 0.10.1.0 and earlier closed consumers without waiting for commits or leave group requests to be sent. With this change, consumer.close() method waits for all pending requests to be sent and hence close() could take longer to complete.

 

Test Plan

Unit tests added in KAFKA-3703 to test the hard-coded timeout will be altered to work with the new close timeout. Existing integration and system tests already test close logic without timeout.

Rejected Alternatives

Use a small hard-coded timeout

A single close method with a small hard-coded timeout can be used to attempt to close gracefully if possible. But this is inconsistent with the Producer where applications can choose to wait for a longer time or not wait at all for graceful close.


  • No labels