Status

Current state: Under Discussion

Discussion thread: here

Vote thread: here

JIRA: KAFKA-16936 - Getting issue details... STATUS

Motivation

Kafka still relies on SLF4J1 as its logging interface, significantly restricting users when selecting logging backends. Users must either ensure their preferred framework appears first in the classpath order or remove other logging frameworks entirely—both approaches require manual JAR file manipulation.

With the introduction of SLF4J2, a new system property -Dslf4j.provider becomes available, allowing users to select logging backends through simple configuration rather than modifying JAR files. 

Note: The rationale for this KIP is that upgrading SLF4J necessitates corresponding provider upgrades, which constitutes a breaking change and should be delivered in Kafka 5.0

Public Interfaces

Update the below files to upgrade slf4j

• dependencies.gradle 
• build.gradle 
• kafka-run-class 

Proposed Changes

1. Upgrade slf4j from 1.7.36 to 2.0.17
   • We will also upgrade the log4j2 dependency based on SLF4J2 (i.e.,  log4j-slf4j-impl to log4j-slf4j2-impl ) 
2. Add a new system variable to the kafka-run-class  (sh & bat) script to define -Dslf4j.provider.
   • By default, we use org.apache.logging.slf4j.SLF4JServiceProvider 
   • reference: https://www.slf4j.org/faq.html#explicitProvider
3. Define the default logging framework for Kafka
   • Log4j2 will be the standard server-side logging framework
   • We expose the log4j2 configuration file (log4j2.yaml) in the Kafka distribution for users to customize

Compatibility, Deprecation, and Migration Plan

• We will also upgrade the log4j2 dependency based on SLF4J2 
• To avoid breaking downstream compatibility, we will not bind the server side to any specific logging framework in the build.gradle, maintaining the current scope - compileOnly  and releaseOnly 
  • https://github.com/apache/kafka/pull/17373#issuecomment-2577813317
• For client-side components, users will need to select their own logging framework, as client-side code typically runs with user applications and should use the same logging framework
• For server-side components, users who wish to use a different logging framework will need to add appropriate jars to the classpath and configure the -Dslf4j.provider  property accordingly
• Version matching between the SLF4J core and providers is critical, as documented in the SLF4J FAQ (https://www.slf4j.org/faq.html#compatibility). Users must ensure the provider version is compatible with SLF4J 2.0.17 when they select their logging framework. 

Test Plan

Imitate the user's behavior by manually adding or replacing the dependent logging jars and assigning the corresponding system property by -Dslf4j.provider .
After that, try starting a Kafka service and see if there are any warnings or errors. Make sure Kafka still usually works after changing the logging framework.

Rejected Alternatives

Delivering this KIP in a minor version

• We cannot ensure SLF4J 2.x would work like 1.x out of the box for client modules. This would potentially break backward compatibility, which violates Kafka's commitment to compatibility in minor releases.
• Solutions involving warnings and classpath ordering do not provide robust solutions to compatibility challenges. Such approaches would introduce fragility and unpredictability to deployment environments, especially in complex production settings where Kafka is commonly used.
• Version matching between the SLF4J core and providers is critical, as documented in the SLF4J FAQ (https://www.slf4j.org/faq.html#compatibility). A major version change like this requires careful coordination that is more appropriate for a major release.

Using java.util.logging

• Kafka already uses SLF4J, and most of the Java ecosystem standardizes on SLF4J as the logging facade. This maintains compatibility with existing configurations and tooling.
• The goal of this KIP is to leverage SLF4J2's provider selection capabilities while minimizing disruption, not to fundamentally change Kafka's logging architecture.