...
- Understand the performance curve for different values of
max.in.flight.requests.per.connection.We expect better throughput and latency for higher values of this variable. But when do the benefits tail off?
- If we want to support max.inflight > 1 when enabling idempotence, should we pick a single value and not allow further configuration? If so, what should this value be?
Understand the effect of acks=all when compared to acks=all1. If it is slower why? Can we make acks=all the default?
...
- Throughput and latency show big improvements from max.inflight=1 to max.inflight=2, but the performance plateaus thereafter.
- Slight throughput degradation between acks=1 and acks=all..
- There is a major 2x degradation in p95 latency between acks=1 and acks=all except for 64 byte messages.
- Plots above are for 9 partitions. If you keep increasing the number of partitions, the difference between acks=1 and acks=all and max.inflight=1 and max.inflight=2 becomes smaller and smaller.
- This not surprising as as the number of partitions increases, the payload of each
ProduceRequestis bigger, hence the relative overhead of additional operations per request is smaller.
- This not surprising as as the number of partitions increases, the payload of each
...