Current state: Accepted
Discussion thread: here
By default Kafka runs with the setting unclean.leader.election.enabled=true. This setting favours availability over durability by allowing an out of sync replica to become leader should there be no insync replica available. Many users run with the default settings. It is typically better to protect the data of those users by default, with the higher availability setting being something that can be enabled if required.
Change the default for the property unclean.leader.election.enabled from true to false
Compatibility, Deprecation, and Migration Plan
Clusters wishing to upgrade, whilst retaining the unclean.leader.election.enabled=true behaviour, will update their server.properties files to include this setting. They should then see no behaviour change when doing a rolling upgrade. The need to make this configuration change, to retain pre-existing behaviour, will be clearly marked in the release notes / upgrade instructions.
We could leave the default value as unclean.leader.election.enabled=true, but change the default version of server.properties to include unclean.leader.election.enabled=false. This would mean that running clusters would not be affected, but new users, who start from the default file, will get the higher durability setting. This was rejected as it is perceived safer and cleaner to change this setting once, consistently.