...
From discussion on the mailing list and within LinkedIn, we now have three options of the end state of timeout configuration for KafkaProduce:
Option 1
...
- max.buffer.full.block.ms - To replace block.on.buffer.full. The max time to block when buffer is full.
- metadata.fetch.timeout.ms - reuse metadata timeout as batch.timeout.ms because it is essentially metadata not available.
- network.request.timeout.ms - This timeout is used when producer sends request to brokers through TCP connections. It specifies how long the producer should wait for the response.
...
- Need to educate user about metadata and buffer.
- The request timeout is less explicit but will be affected by retries, linger.ms, retry backoff time, request timeout, etc.
Option 2
...
- max.enqueue.block.ms - the maximum time producer.send() will block, including:
- metadata fetch time
- buffer full block time
- serialization time (customized serializer)
- partitioning time (customized partitioner)
- request.timeout.ms - the maximum time to wait for the response of a message after message has been appended to the accumulator. Including:
- linger.ms
- actual network RTT
- server replication time
- retries
...