Current state: Voting
Discussion thread: here
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
In KAFKA-4879, we first noticed that KafkaConsumer will hang forever when deleting a topic. However, after looking closer, we have found that in general the
position(TopicPartition topicPartition) ,
committed() , and
updateFetchPositions() which blocks through a series of callbacks) method will continue to block indefinitely unless the offsets are retrieved for the provided
TopicPartition. To avoid this scenario from occurring:
- A secondary method will be added, but with an extra parameter
timeoutwhich bounds the amount of time spent.
TimeoutExceptionwill be thrown once the amount of time spent exceeds
TimeoutException will be thrown when the time spent exceeds
Similarily, this will also be applied to other methods in KafkaConsumer which will block indefinitely.
A TimeoutException will be thrown if time spent of
position() method exceeds
Compatibility, Deprecation, and Migration Plan
Since old methods will not be modified, preexisting data frameworks will not be affected. However, these methods might be deprecated in favor of methods which are bound by a specific time limit.
One alternative was to add a
timeout parameter to
position() and other methods. However, the changes made by the user will be much more extensive then basing the time constraint on
requestTimeoutMs because the method signature has been changed.
Another possibility was the usage of
requestTimeoutMs to bound
position(), however, this would make the method highly inflexible, especially since
requestTimeoutMs is already being used by multiple other methods.