Status

Current state: "Accepted"

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

There are many Flink contributors and fewer committers. As a community we want to drive down our technical debt. The committers can be time strapped, so may not have enough time to review every PR that comes in. There are members of the community that are willing to review PRs and appropriately share the review burden. This activity has been occurring under the Community Health Initiative, where every new PR since the start of the group has been reviewed / triaged. This proposal is to formally create a process where the community (non-committers) can review PRs and ease the load on the committers. The benefits of this approach are:

• we formally encourage the community to review PRs - making this one of the things we do as a community.
• encouraging the community to engage in reviewing PRs will get more eyes on code changes.
• easing the burden of review for committers:
  • for straight forward PRs that the community approves of
  • identify PRs if they need expert assessment
  • feeding back on code contributions to improve contribution content before it gets to a committer
• community reviewing then becomes a respected way to contribute to Flink on the road to becoming a committer. So, it is in the contributor’s interest to review.
• measuring community review activity gives us metrics to show its impact and whether the process is working.
• it is easier for the PMC to screen for committer candidates based on their review involvement and quality.

Public Interfaces

• GitHub Action

Proposed Changes

1) Update the wiki process to add a section on the community review.

2) Create a GitHub Action  to manage new community review labels

Implement a GitHub Action to drive a bash script that will be driven when a PR is reviewed. It will :

• find the current user
• check whether they have write access to the repository to determine if they are a committer or not. 
• check if the PR has 2 or more approving reviews from non-committers, set community-reviewed-LGTM
• if the label is not set and the reviewer is not a committer set label community-reviewed. This will be set for all review types not just approvals, as negative and comment reviews should be tracked as they are often more valuable than approvals.

The script will look to ensure there is no more than one label related to community review - so it does not overpower the PR with multiple labels relating to community view

The flow diagram of the proposed GitHub Action script is:

We would like to run this script or an amended version of it against historical data.  

This script can be tested on a forked repo.

The GitHub Action needs to run with committer access.

We are open to add more labels if we think they will ease the process and bring down the technical debt, but this is a starter set of labels indicating our intent around community health improvement.  

Value:
The by-produce of this process is that sanity checks (does the Jira have a title – look right, does it have unit tests, code logic tests, does the Jira have a decent description on what is being changed and why etc) will occur on PRs generating lots of prompt feedback to the submitters of the PRs. 

Also it is easier for committers to identify what PRs to review and merge easy changes .
In the Community Health Initiative, we hope to move as much of the sanity checking as possible to the GitHub Actions, where it can be automated.

Compatibility, Deprecation, and Migration Plan

• n/a

Test Plan

We will manually test the GitHub Actions and labels are working correctly. 

Rejected Alternatives

1) Flinkbot

To manage the following labels via new Flinkbot commands. 

With the following new Flinkbot commands:

We talked about adding a label, but were not convinced about the value, so thjis is currently out of scope

*   community-reviewed-requires-deep-review
This could be set if 2 community members agree that a deep review is required on a PR using a flinkbot command

A committer could then search for this label and the components they are interested in to find a list of appropriate PRs to review.

2) Only set the community review label for approves