Status

Current state: Under Discussion

Discussion thread: here

JIRA: KAFKA-16437 

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Older versions of JavaEE that use the legacy javax package have long been deprecated since the donation to Jakarta which has now become Jakarta EE. 

The new package structure for Jakarta EE  uses jakarta and is not compatible with the legacy javax package namespace and this causes issues when trying to mix the two.  Kafka 4.0 should upgrade and modernize to the new APIs.

Besides the obvious reason of upgrading to keep up with the latest changes, a few other reasons to upgrade include;

  1. Kafka 4.0 is a major version and a good time to make the upgrade since it requires incompatible changes to packages.
  2. Several popular projects and frameworks have moved to Jakarta such as Jetty 12, Spring 6, Spring Boot 3, Apache Tomcat 11, ActiveMQ 6. Many others will be upgrading as well.
  3. Upgrading allows keeping up to date with dependencies such as Jetty to receive bug fixes and security fixes.
  4. Anyone that runs Kafka as an embedded broker (such as for unit testing) will run into problems if relying on new libraries that have upgraded to Jakarta.

There are a few additional considerations for the Jetty version:

  1. Jetty 11 could be used as it still supports JDK 11 (which the Connect framework is still using), however that version is now EOL. Jetty 12 requires Java 17 so if need to keep compatibility with JDK 11 for the Connect framework then we need to stay with an EOL jetty version which has risks. Jetty 11 for now is receiving security updates still but there is no guarantee that will be long term. 
  2. This is a major version upgrade so making the jump to requiring Java 17 now for Connect seems like it would make sense because if we don't do it now we would likely be unable to do so until Kafka 5.0, which could mean ending up relying on an EOL version of Jetty for a long time.
  3. If we use Jetty 12 we can use JavaEE 10 but Jetty 11 only supports JavaEE 9

Public Interfaces

There shouldn't be changes to the main client API as the API doesn't use jakarta. The primary changes will involve Jetty and the REST service for the Connect framework.

Any projects that rely on the Kafka classpath/dependencies (such as a build that pulls in dependencies transitively or a connector) will need to be upgraded. Connectors can use their own classpath and isolated classloader.

Proposed Changes

Several legacy dependencies that use the javax package will need to be upgraded to the new versions that use the jakarta namespace. Anything that relies on the old namespace must be updated.

This includes libraries such as:

  • Upgrading Jetty to 11.x or 12.x. Version 11.x still supports JDK 11+ but is EOL (except for security updates). Version 12.x requires JDK 17+
  • Jersey needs to be upgraded to version 3.x
  • Adding new/upgrading related apis such as jakarta.servlet-api, javax.ws.rs, javax.activation, javax.xml.bind
  • jackson-jaxrs-json-provider is needed for compatibility with jakarta.ws.rs-api

It's possible there are other things that need to be updated as well that will be discovered as the work is being done. This KIP could be updated if other things are discovered.

Note: There is an existing PR KAFKA-10176 that was closed that could be re-opened as a starting point for the changes.

Compatibility, Deprecation, and Migration Plan

  • Users who use Kafka as a stand alone install would likely not be impacted as internally all the APIs and Jetty would be upgraded so it would just be a normal upgrade.
  • Users that depend on Kafka dependencies in a build (such as a Maven build) will need to upgrade their build to be compatible with the new jakarta based dependencies if they are pulled in transitively.
  • Users that use the Connect framework will need to upgrade their custom connectors to use the new jakarta namespace (if they rely on it) or else will need to use an isolated classloader.
  • JDK 17+ will be required if Jetty 12.x is chosen to be used (just like the Kafka 4.0 broker will require JDK 17).

Test Plan

Testing will be done by running the existing test suite and making sure everything still works after the upgrade to the new jakarta dependencies.

Rejected Alternatives

  1. Continue supporting legacy javax namespace and dependencies
  • No labels