Current state: Under Discussion
Discussion thread: here
JIRA: here [Change the link from KAFKA-1 to your own ticket]
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
KIP-482 added support optional tagged fields. Tagged fields require no space when they're omitted from RPCs using flexible versions.
A 16 bit
ErrorCode is used in the vast majority of RPCs and will normally have value 0 (NONE). Similarly
ErrorMessage will in these cases be null, but still requires a byte to store the string length.
By changing these mandatory fields to tagged fields we can save two bytes for every error code and 1 byte for every message.
Since some RPCs have error codes and messages for each batched entity in a response the saving can be more significant across a whole message.
When the error code is nonzero a tagged field will require 4 bytes (tag and size each 1 byte varints and the 16 bit value requires two bytes), rather than the 2 bytes needed for an
Similarly non-null error messages will require 2 bytes (tag and size each 1 byte varints) more than the usual serialization of the string.
This KIP proposes to use tagged fields for error codes and messages in almost all RPCs in order to optimize the usual case.
Change every RPC which has an
SASL_HANDSHAKE, due to KAFKA-9577) as follows:
- Increment the RPC version (both request and response), since we're making a previously required field optional
- If the RPC is not using
flexibleVerionsthen add it (just
Compatibility, Deprecation, and Migration Plan
This change is backward compatible with existing clients.
Doing this for each RPC organically. This would mean a (long) period of inconsistency in how these fields are handled, would complicate other KIPs and would be easily forgotten about.