Status

Current state: Accepted

Discussion thread: here

JIRA: KAFKA-16437 

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Older versions of JavaEE that use the legacy javax package have long been deprecated since the donation to Jakarta which has now become Jakarta EE

The new package structure for Jakarta EE  uses jakarta and is not compatible with the legacy javax package namespace and this causes issues when trying to mix the two.  Kafka 4.0 should upgrade and modernize to the new APIs.

Besides the obvious reason of upgrading to keep up with the latest changes, a few other reasons to upgrade include;

  1. Kafka 4.0 is a major version and a good time to make the upgrade since it requires incompatible changes to packages.
  2. Several popular projects and frameworks have moved to Jakarta such as Jetty 12, Spring 6, Spring Boot 3, Apache Tomcat 11, ActiveMQ 6. Many others will be upgrading as well.
  3. Upgrading allows keeping up to date with dependencies such as Jetty to receive bug fixes and security fixes.
  4. Anyone that runs Kafka as an embedded broker (such as for unit testing) will run into problems if relying on new libraries that have upgraded to Jakarta.

There are a few additional considerations for the Jetty version:

  1. Jetty 12 is the current version of Jetty and requires JDK 17 so Connect/MM2 will need to be bumped to require JDK 17, otherwise we would need to stay with an EOL jetty 11.x version which has risks.  Jetty 11 for now is receiving security updates still but there is no guarantee that will be long term. 
  2. This is a major version upgrade so making the jump to requiring JDK 17 now for Connect/MM2 makes sense because if we don't do it now we would likely be unable to do so until Kafka 5.0, which could mean ending up relying on an EOL version of Jetty for a long time.
  3. KIP-1013 has already established a precedent for moving the required JDK version to 17+ for the broker itself for Kafka version 4.0 so this would bring the other server components in line.
  4. Jetty 12 supports both JavaEE9 and JavaEE 10 but it makes sense to use the newest.

Public Interfaces

The primary changes will involve Jetty and the REST service for the Connect framework and MM2. Any projects that rely on the Kafka classpath/dependencies (such as a build that pulls in dependencies transitively or a connector) will need to be upgraded. Connectors can use their own classpath and isolated classloader.

For the client API, the only change will be to the REST extension point inside the connect-api module, ConnectRestExtensionContext. The method ConnectRestExtensionContext::configurable will change to return an instance of jakarta.ws.rs.core.Configurable instead of javax.ws.rs.core.Configurable.


Jakarta version of ConnectRestExtensionContext.java
package org.apache.kafka.connect.rest;

import org.apache.kafka.connect.health.ConnectClusterState;

import jakarta.ws.rs.core.Configurable;

/**
 * The interface provides the ability for {@link ConnectRestExtension} implementations to access the JAX-RS
 * {@link jakarta.ws.rs.core.Configurable} and cluster state {@link ConnectClusterState}. The implementation for the interface is provided
 * by the Connect framework.
 */
public interface ConnectRestExtensionContext {

    /**
     * Provides an implementation of {@link jakarta.ws.rs.core.Configurable} that can be used to register JAX-RS resources.
     *
     * @return the JAX-RS {@link jakarta.ws.rs.core.Configurable}; never {@code null}
     */
    Configurable<? extends Configurable<?>> configurable();

    /**
     * Provides the cluster state and health information about the connectors and tasks.
     *
     * @return the cluster state information; never {@code null}
     */
    ConnectClusterState clusterState();
}


Proposed Changes

Several legacy dependencies that use the javax package will need to be upgraded to the new versions that use the jakarta namespace. Anything that relies on the old namespace must be updated.

This includes libraries such as:

  • Upgrading Jetty to the latest 12.x version. Version 12.x requires JDK 17+ and supports JavaEE 10
  • Bump the minimum version of Connect and MM2 to JDK 17+
  • Upgrade to JavaEE 10 and Servlet 6.0 API
  • Jersey needs to be upgraded to version 3.x
  • Adding new/upgrading related apis such as jakarta.servlet-api, javax.ws.rs, javax.activation, javax.xml.bind
  • jackson-jaxrs-json-provider is needed for compatibility with jakarta.ws.rs-api

It's possible there are other things that need to be updated as well that will be discovered as the work is being done. This KIP could be updated if other things are discovered.

Note: There is an existing PR KAFKA-10176 that was closed that could be re-opened as a starting point for the changes.

Compatibility, Deprecation, and Migration Plan

  • Users who use Kafka as a stand alone install would likely not be impacted as internally all the APIs and Jetty would be upgraded so it would just be a normal upgrade.
  • Users that depend on Kafka dependencies in a build (such as a Maven build) will need to upgrade their build to be compatible with the new jakarta based dependencies if they are pulled in transitively.
  • Users that use the Connect framework will need to upgrade their custom connectors to use the new jakarta namespace (if they rely on it) or else will need to use an isolated classloader.
  • JDK 17+ will be required going forward for Connect and MM2 with the Jetty 12.x upgrade (just like the Kafka 4.0 broker will require JDK 17).

Test Plan

Testing will be done by running the existing test suite and making sure everything still works after the upgrade to the new jakarta dependencies.

Rejected Alternatives

  1. Continue supporting legacy javax namespace and dependencies
  • No labels