This page is meant as a template for writing a KIP. To create a KIP choose Tools->Copy on this page and modify with your content and replace the heading with the next KIP number and a description of your issue. Replace anything in italics with your own description.
Current state: Under Discussion
Discussion thread: here [Change the link from the KIP proposal email archive to your own email thread]
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Consumer API currently has one close() method without a timeout. KAFKA-3703 added graceful close of the consumer with a hard-coded timeout of 5 seconds to ensure that consumers complete pending commits and leave the group gracefully before closing connection to the coordinator.
Producer interface has two close() methods.
The first variant waits for graceful close without a time limit (
timeout=Long.MAX_VALUE). The second attempts to close gracefully, but terminates the produce, aborting any pending requests after the timeout.
For consistency with producers, consumers should also expose the same two variants for close.
A new close method with timeout will be added to the
Consumer interface. The existing
close() method will attempt to close gracefully
close() method without a timeout will attempt to close the consumer gracefully with a timeout of
Long.MAX_VALUE instead of the hard-coded 5 second timeout. The current close logic that attempts to commit offsets if the coordinator is known and leave group will remain the same but will be executed with the specified timeout instead of the hard-coded 5-second timeout.
Compatibility, Deprecation, and Migration Plan
What impact (if any) will there be on existing users?
Versions 0.10.1.0 and earlier closed consumers without waiting for commits or leave group requests to be sent. With this change,
consumer.close() method waits for all pending requests to be sent and hence
close() could take longer to complete.
Unit tests added in KAFKA-3703 to test the hard-coded timeout will be altered to work with the new close timeout. Existing integration and system tests already test close logic without timeout.
Use a small hard-coded timeout
A single close method with a small hard-coded timeout can be used to attempt to close gracefully if possible. But this is inconsistent with the
Producer where applications can choose to wait for a longer time or not wait at all for graceful close.