Status

Current state: "Under Discussion"

Discussion thread: here

JIRA: None yet

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Before Kafka 2.8.0 all KIPs seemed to be released without any type of qualification into the project. During 2.8.0 it was the first time that a KIP was released with "early access". This was done in the context of KRaft as a replacement for ZooKeeper (https://lists.apache.org/thread/m16d7om3lry70zw9b1t6fyw99tlppkd3). However, we never really defined what "early access" meant, for the developers nor for the end users.

An older instance of an intention of releasing a KIP in a qualified mode was Tiered Storage (https://lists.apache.org/thread/w5w55jyko9zvh3fxxh4lrs6z7trx3nzk). Here, "Preview" was used as a qualifier to signify that the KIP wouldn't be complete and the target was a release in the future.

Since then, we have released several KIPs into Kafka with either "Early Access" or "Preview" qualifiers, sometimes a KIP even went through "Early Access" first and "Preview" later (https://github.com/apache/kafka-site/pull/619).

We need to agree on the number of steps and what each of these steps mean for developers as well as for end users of said KIPs.  We had some discussion in the mailing list and gathered some feedback here: https://lists.apache.org/thread/5z6rxvs9m0bro5ssmtg8qcgdk40882bz

The proposal is to create 4 levels, 1 being the lowest and 4 being the most mature level. By having these levels, the release manager can easily know if KIPs are present in a release (and under which category) of if they need to be postponed to the next one. Only KIPs that are declared level 2 and onwards can be released in a given release, KIPs that remain in Level 1 are to be pushed to further releases.

During the discussion phase of a KIP, the community needs to agree if the KIP itself is complex enough in order for this graduation steps to apply.

Public Interfaces

No changes in public interfaces

Proposed Changes

Graduation Steps

In order to reduce bike-shedding on the names of the steps, I propose to first use placeholder ones to define the hierarchy of steps and the implications for each of them. This step system is not meant to ensure a bug-free KIP (more than we already try to ensure), but rather express the level of community input needed for a given KIP.

Level 1

This is the "farther" from production a KIP is. The KIP is not yet completed by any means and it's not yet usable. For completion sake, in order to define graduation steps of a KIP, we need to start by step 0 which is when the KIP is being made.

The KIP is not yet stable and it is not meant to be used by end users.

Level 2

The KIP is now usable, but it might not be completed yet. This is intended for bigger KIPs that are delivered in multiple milestones. As long as the KIP reaches a milestone that enables end users to use the KIP, its developers can decide to graduate from level 1 to level 2.

A KIP in level 2 should be usable for end users, it should contain complete flows and it shouldn't, knowingly, leave Kafka clusters in any type of corrupt state. It might be that only a subset of the functionality is available or implemented. End users are encouraged to start testing this KIP and to provide feedback and bug reports for it.

The API for this KIP is not to be considered stable, developers reserve the right to change the API to fix any kind of bugs or inconsistencies found during testing. It is not encouraged to build tools or applications that would rely on these APIs. The fact that the API can change doesn't exempt from updating the KIP and submitting it for a new VOTE thread if needed. The KIP shouldn't be enabled by default, and users might need to enable experimental flags to use it.

This level is optional, and a KIP does not need to go through it to reach the next level.

Level 3

Mildly complex KIPs might reach this level directly from level 1, while more complex ones would reach it from level 2.

A KIP in level 3 should be feature-complete (as per the KIP that defined it), it can still be evolved in terms of, for example, adding more and better metrics, documentation, or improving certain usability aspects of the KIP. The KIP is intended to work in different types of production environments, but there might be some aspects that might need some more testing and validation. The purpose of this level is to request help from the community to test this KIP under said scenarios to confirm the KIP works as expected. For example, help might be needed to test the KIP in extreme high volume environments.

The API for this KIP is to be considered stable and it can't change without undergoing a full deprecation cycle (at least 12 months in deprecated state after being removed in a "major" release). The KIP shouldn't be enabled by default.

This level is optional, and a KIP must not go through it in order to reach the latest level. Any KIP set at this level is encouraged to remain in it for 2 releases before graduating to level 4. This is just a recommendation.

Level 4

This is the final stage of the KIP. Any KIP might reach to this level directly, but it's only recommended for simpler and smaller KIPs.

A KIP in level 4 should be feature-complete (as per the KIP that defined it) and it must have full documentation, test suites and if it applies, system tests. If the KIP went through "Level 3", all potential problems found are addressed. This level doesn't offer any more guarantees that any other KIP that is already released in Kafka, bugs might still occur.

The API for this KIP is to be considered stable and it can't change without undergoing a full deprecation cycle (at least 12 months in deprecated state after being removed in a "major" release). The KIP might be enabled by default (not all new functionalities need to be enabled by default)

Table

[1]: Implicitly when a KIP is approved and development starts, the KIP is effectively in "Level 1"

[2]: As a final state of graduation all KIPs aspire to get to this level.

Examples

After KIP-853: KRaft Controller Membership Changes got approved, it would have been positioned at Level 1. It stayed in that level for a while. In between, Kafka 3.8 release happened and it remained at Level 2. After that, it got promoted to Level 2 just in time to be taken in Kafka 3.9 release.

KIP-848: The Next Generation of the Consumer Rebalance Protocol on the other hand reached Level 2 at the time of Kafka 3.7. Improvements were done after that and when it was time to release Kafka 3.8 the KIP got promoted to Level 3.

An example of a KIP in Level 4 could be KIP-390: Support Compression Level that reached this level during Kafka 3.8, and jumped directly from Level 1 to Level 4.

Names

1. "In Development"
2. "Early Access"
3. "Preview"
4. "Production Ready"

Alternatively, if we want to be a bit more playful we could go with a theme borrowed from the book industry (as an homage to Franz Kafka):

1. "In Development"
2. "Manuscript"
3. "Pre-print"
4. "Published"

When can a KIP change levels?

KIPs can only move in one direction on the step system, level 1 to level 4, and can't go back to the previous level. A KIP can be promoted to a "higher" level at minor and major releases (not patch releases).

Documentation

A new page in the documentation page will be created containing the agreed graduation steps for KIPs and the implications for each of them. This new page should be under "Key Concepts" (also known as "Getting Started") in its own section. Right now it would be "1.7 Graduation Steps for KIPs".

How to track level changes

KIPs that require to go through the different graduation levels, must include a new section (Graduation Steps Log) containing the change log for the graduation steps and their date/release.

When preparing a release plan, the release manager should look at this section to capture the KIPs state.

If the KIP changes graduation step while a release is being prepared, the driver of the KIP should communicate this in the release thread.

Each graduation step can be stated by the developer "owning" the KIP. The developer might decide to seek consensus via a VOTE thread to promote to level 3 and 4.

Currently, KIPs have a "Current state" field that contains the following values: "Under Discussion", "Voting", "Rejected" and "Accepted". This KIP proposes to update this field with the right graduation level once the KIP reaches it.

Definition of "Usable"

During this KIP "usable" is being used extensively in this proposal. Having a really prescriptive definition for "usable" might probably exclude certain types of KIPs to feel identified with this process. This is why we could understand "usable" in a similar way that "Minimum Viable Product" defines it:

Usable refers to a product or version that delivers enough functionality to allow users to achieve their primary goals.

In our scope, usable means that a KIP has enough parts implemented that the end users can start testing it and providing feedback even though the KIP might not be yet feature complete.

Compatibility, Deprecation, and Migration Plan

Does not apply

Test Plan

Does not apply

Rejected Alternatives

The use of "early availability" and "general availability" have been rejected because, this being an open source project, all KIPs are by default available, Kafka is not ruled by a centralized entity that decides which users get which KIPs. "Early availability" and "general availability" give away the impression that KIPs are not able to be used by end users until we decide to toggle some switch.

Using "alpha", "beta" and "gamma" has been rejected because of the widespread use of "alpha" and "beta" being used as a qualifiers for software versions not features within a software. For example the Wikipedia page: https://en.wikipedia.org/wiki/Software_release_life_cycle. Talking about alpha and beta might create confusions about users with regards of the stability of the Kafka version itself.