Child pages
  • KIP-266: Add TimeoutException to KafkaConsumer#position()
Skip to end of metadata
Go to start of metadata


Current state: Voting

Discussion thread: here

JIRA: here

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).


In KAFKA-4879, we first noticed that KafkaConsumer will hang forever when deleting a topic. However, after looking closer, we have found that in general the position(TopicPartition topicPartition) ,commitSync()committed() , and poll() (since pollOnce() calls updateFetchPositions() which blocks through a series of callbacks) method will continue to block indefinitely unless the offsets are retrieved for the provided TopicPartition.  To avoid this scenario from occurring:

  1. A secondary method will be added, but with an extra parameter timeout which bounds the amount of time spent.
  2. A TimeoutException will be thrown once the amount of time spent exceeds timeout.

Public Interfaces

TimeoutException will be thrown when the time spent exceeds requestTimeoutMs:

KafkaConsumer#position(TopicPartition topicPartition))

Similarily, this will also be applied to other methods in KafkaConsumer which will block indefinitely.

Proposed Changes

A TimeoutException will be thrown if time spent of position() method exceeds requestTimeoutMs.

Compatibility, Deprecation, and Migration Plan

Since old methods will not be modified, preexisting data frameworks will not be affected. However, these methods might be deprecated in favor of methods which are bound by a specific time limit.

Rejected Alternatives

One alternative was to add a timeout parameter to position() and other methods. However, the changes made by the user will be much more extensive then basing the time constraint on  requestTimeoutMs because the method signature has been changed. 


Another possibility was the usage of requestTimeoutMs to bound position(), however, this would make the method highly inflexible, especially since requestTimeoutMs is already being used by multiple other methods.

  • No labels