...

New segments written as part of log compaction will always use message format v2. Note that this was originally specified in the KIP to happen in Apache Kafka 3.0, but it was postponed to 4.0. There are subtle correctness considerations when up-converting records that had been previously written with an old format. Replication is aligned by batches, but up-conversion will happen independently on each broker for case 3. For compressed records, v0 and v1 message formats support batching, so it's straightforward. Uncompressed records, however, are not batched in v0 and v1. To ensure alignment, we will convert them to single record batches in v2. It's worth noting that message format v2 is slightly less efficient than v0 and v1 when single record batches are used, but it's an acceptable cost for correctness and it only impacts older records. Over time, these records will either be deleted via retention or will be replaced by new versions for compacted topics.

Compatibility, Deprecation, and Migration Plan

...