This page describes a Request For Comments (RFC) process for proposing a major change to Hudi or even just sharing designs/vision for the project to get early feedback.

To create your own RFC, go to the RFC Template and copy it as follows.

Be sure, to enter the RFC number at the top of the copied page before saving, to generate the correct url for the page.

When does a feature require a RFC ?

Whenever a feature is considered to be causing a “major change” in the project, such a feature requires a RFC. Any of the following can be considered a major change:

  • Any new component, module or code that introduces a new concept into the project or alters the behavior of an existing one

  • Any large code refactor to address general code reusability and structure. There is no strong definition for “Large” and whether or not the refactor requires a RFC can be discussed on the @dev mailing list.

  • Any change that impacts the underlying file format. More specifically changes to the HoodieLogFormat.

  • Any changes to the index of the dataset

    • HoodieBloomIndex and associated classes

    • HbaseIndex

  • Any change that impacts the public interfaces of the project listed below

    • Spark datasource usage of Hudi

    • HoodieWriteClient or HoodieReadClient

    • HoodieDeltaStreamer

  • It can also be used to describe large direction shifts to the project (e.g: Flink support) or new green field ideas (e.g Hudi for ML flows)

Not all RFCs require the same effort and detail. For critical changes such as the File Format and Index, we need to deeply discuss the trade-offs of making such a change and how it impacts current and new users. Any changes to such components affect the correctness of a dataset (backwards and forward versions supported). Other changes such as code refactor might require more details around abstractions but as long as there is good test coverage, a migration plan section can be avoided. It may happen that you are making a bunch of changes across many components to enable an already existing feature. For example, introducing a new config along with reporting metrics, enhancing a tool and also improving documentation and the onboarding experience. If all of these changes are linked to a general feature/idea, these can be grouped together under a single RFC.

Who should initiate the RFC ?

Anyone can initiate a RFC. Please note that if you are unsure of whether a feature already exists or if there is a plan already to implement a similar one, always start a discussion thread on the dev mailing list before initiating a RFC. This will help everyone get the right context and optimize everyone’s usage of time.

How do I initiate a RFC ?

Here is the process for making a RFC:

  1. Start a discussion thread on the Apache Hudi dev mailing list. The subject of the discussion thread can be DISCUSS [proposed idea] to create a RFC. Use this discussion thread to get an agreement from people on the mailing list that your proposed idea necessitates a RFC.

  2. Take a RFC number (at the bottom of this page) and use the template above to create a new RFC and fill up all the applicable sections. Update/Create the discussion thread with title "[DISCUSS] RFC [RFC number] [RFC title]"

  3. Update the RFC number (increment by 1) at the bottom of this page.

  4. Once the proposal is finalized, add at least 2 of the PPMC members as approvers of your document. You are free to add any number of dev members to your reviewers list.

  5. Work through feedback until all the questions and issues are addressed and the RFC approved by the approvers.

RFC round-up

Next RFC Number: 35

Use this number as the identifier for your RFC and increment this value by 1