WORK IN PROGRESS
Email Context
Nathan sent this email which is a reasonable outline. So I have it here for a reference to work toward, I will be making some changes where I think are appropriate. At the end of the day this probably should go into a CONTRIBUTING file on GitHub.
At this point, much has been written in this and other threads. Now I recommend that we should have a Confluence page where the information can be edited to make it: * Plain English * Flow well * Readable * Coherent * Specific * Not assume knowledge * In one easy to find place I say "I recommend that we should" instead of just going ahead and creating that Confluence page because I want to ensure that this happens with community agreement. Rationale for creating the document: I don't want or expect anyone to memorize our workflow, nor to go digging all over the place to piece together what it is. I want it documented clearly so that veteran contributors and total newbies alike will be able to understand exactly what to do. I also want this documented for purpose of on-boarding future committers and so that committers will know what is expected of them whenever they commit. And furthermore I want it in a clear document that we can officially vote on to make it our "blessed" workflow, until such time as we decide to make changes and vote to update the "blessed" process. The document should flow in a manner that makes information easy to find. I suggest to organize it as follows: 1. Overview. This section defines "WHAT" the steps of the workflow are. This is a simple bulleted or numbered list. No rationale, no git jargon, no DevOps nonsense, just clear plain English. Where is the code. What basic steps take place to apply changes to it. Greg literally wrote this list a few emails ago and it contains 80% of what should be in that list. It needs minimal improvement. 2. How To Submit Changes For Review. This section defines the "HOW" of the workflow for ANYONE who wants to get a change into NuttX, whether committer or not. Committers do not get to bypass this process. This section should document: How to obtain a copy of the code. What steps to take before beginning work on a change. Once you've made your change and want to contribute it, what steps to take to turn it into a PR or patch. How to submit the PR or patch to us? We should NOT assume knowledge. If a step requires using git, then give the exact git command followed by an explanation of every element of that command, so that anyone who knows how to enter a command in a terminal with zero prior knowledge of git will be able to understand exactly how to issue that git command and exactly what it will do. I want to make it straightforward and easy for a HARDWARE engineer to be able to submit changes. 3. Criteria For Acceptance. This section defines what sorts of things committers will examine and verify before allowing changes into NuttX. First, the universal requirements that apply to all parts of NuttX. This includes INVIOLABLES, coding standard, rules that govern clean architecture, POSIX compliance, not breaking the build, etc. The word INVIOLABLES should be a link to that file. After the universal requirements, there should be requirements by area. Requirements for boards. Requirements for drivers. Requirements for the scheduler. Requirements for architectural support. Etc. All of the checks in this section can be performed manually by a committer for now. This section should be documented clearly and specifically enough that it can directly be used as the specification of automated checks to implement in a CI system. 4. Reference For Committers. This section explains to committers how to carry out all the steps to process a proposed change from start to finish. "Start" means a patch or a PR arrived. If a patch, how to convert it into a PR. "Finish" can mean either applying the change to master, or reject the change / send it back to the submitter for additional work. And all steps in between. This section (item #4) is a continuation of item #2. Item #2 explained how anyone submits a proposed change for review. Now we explain what happens next. Like item #2, we should NOT assume knowledge. If a step requires using git, then give the exact git command followed by an explanation of every element of that command. This is not excessive. This will help on-boarding of new committers as well as help veteran committers avoid mistakes. 5. Rationale. This section explains the "WHY" behind all of the above. So that people won't cut an inch off each end of a ham without knowing why. Cheers, Nathan
From My Email (will drop this after I fill in the rest)
Submit a PR on GitHub against master if it is approved by one commiter (that did not propose it) it can be merged. The approval is done via the GitHub approval system.
A commiter may create a PR on behalf of a patch submitted to the mailing list.
Commiters can ask for others to review or approve. But at the end of the day they are the ones who approve and merge.
Overview
Purpose
The purpose is simple: We'd like to help everyone participate in the future of NuttX by providing clear and open documentation on how to contribute.
This document can be declared a success if newbies and veteran NuttX developers alike find it valuable as both a how-to and a reference. We hope you'll find it easy to read and that it eliminates the need to hunt, memorize, or guess about parts of the workflow.
Enthusiastic Contributors Welcome!
NuttX is a free open-source project. If you'd like to participate, whether it's to enhance documentation (even this workflow document) or dive into the nitty gritty of some low-level drivers, please read on!
Also, join the conversation at our dev mailing list by emailing dev-subscribe@nuttx.apache.org. The mailing list is open to the public and archived. You can browse older messages at https://lists.apache.org/list.html?dev@nuttx.apache.org.
Why Contribute to NuttX?
Have you fixed a bug or implemented a new feature in your copy of NuttX? If so, contributing your improvements "upstream," which means getting them included in the official NuttX code, may be in your best interest because it means that you won't have to re-fix, re-implement, or port your changes over when we release a newer version of NuttX. It also increases the likelihood of your code getting more exposure and testing, and with that, the possibility of other people contributing additional improvements to your code, which will benefit you "for free."
Who Are The Players?
- Contributor: Anyone who wants to participate in NuttX by contributing improvements.
- Committer: A person who can commit improvements directly to the codebase.
- PMC: Project Management Committee. (Currently, NuttX is a Podling of the Apache Incubator, so currently this is called the PPMC.)
- Upstream: In this document, "upstream" refers to the NuttX project.
- Downstream: In this document, "downstream" means anyone who packages, customizes, and/or uses NuttX in their own projects.
Where is The Code?
Sources of NuttX Code
NuttX source code is available in two forms:
- As a clone or fork from a NuttX GIT repository, or
- As a versioned release in the form of a GZIP'ed tarball
The versioned release tarballs contain no SCM information. They are intended for use with whatever SCM you select in your custom environment.
Versioned GIT Branches
The code on which versioned releases are based is also available within the NuttX GIT repository clones or forks. Historical releases are identified with tags. For example, the tag nuttx-8.2 identifies the nuttx-8.2 release. A versioned branch could be created with the git command:
git checkout -b <branch-name> nuttx-8.2
The code that you will find on the branch will differ from the the code in the versioned tarballs in two ways:
- It will contain the complete GIT history leading up to the versionsed release, and
- It will not contain the .version file. That version file exists only in the versioned tarball releases.
REVISIT: We do not yet know the form of releases for future Apache releases. These will most likely already be on a branch and will include the .version file on that branch. (Note: The release process should be discussed in a different section, or even a completely separate document.)
NuttX GIT Repositories
The Apache NuttX project uses Git SCM as its version control system.
There are two primary repositories:
- nuttx - The real time operating system itself.
- apps - Optional applications for the operating system. You can use this repository or make your own custom one.
The main "Single Source of Truth" repositories are hosted by the ASF:
- https://gitbox.apache.org/repos/asf?p=incubator-nuttx.git
- https://gitbox.apache.org/repos/asf?p=incubator-nuttx-apps.git
These are synchronized with repositories hosted at GitHub:
Released Code Tarballs
REVISIT: We do not know the form or location of future Apache NuttX releases.
Apache project releases are found at https://projects.apache.org/releases.html or https://archive.apache.org/dist/. However, podling releases are handled differently. They are released using the Github release system:
- https://github.com/apache/incubator-nuttx/releases
- https://github.com/apache/incubator-nuttx-apps/releases
REVISIT: The above releases are not the official NuttX releases. They do not include the .version file. I am not sure how those were generated but they are incorrect.
Perhaps I am confused. Perhaps the releases will be at https://archive.apache.org/dist/incubator/incubating-nuttx/ which does not yet exist.
REVISIT: This "Overview" section makes no mention of the buildroot, tools, and uclibc repositories. We do not yet know how/where those will be hosted.
The Workflow
All contributors, whether seasoned NuttX committers or new contributors, follow the same steps to submit a proposed change.
First, we will outline the basic steps. Afterwards, we will go into detail on each step.
All Contributors:
- Optional: Before you begin work, discuss your proposed change on the dev@nuttx.apache.org mailing list. Someone else may already be working on a similar change and perhaps you could collaborate and split the effort. Also, a discussion could help you find out whether your changes are likely to be accepted, and input from other community members might provide some valuable insights.
- Review the section "Criteria For Acceptance" in this document.
- Obtain a copy of the source code, if you don't already have it.
- Develop your changes.
- Submit your changes.
Community/Committer Review:
- At this point, the community, especially committers, should review your proposed changes to ensure that the project's criteria are met. These are spelled out in the section "Criteria For Acceptance."
- If any issues are found, you may be asked to correct them and re-submit your changes. Don't be discouraged if this happens; it's a normal part of the open source process. Committers may opt to go ahead and correct minor issues (such as coding style, etc.) instead of sending them back to you.
- Once the review has taken place and the project's criteria are met, your changes are ready to be committed to the official repository.
Notes:
- No one is obligated to review a change. Generally, committers will review changes that interest them or affect areas they care about. Fortunately, the NuttX community is friendly and has a large following, so chances are pretty good that your changes will be reviewed. (You did discuss your ideas on the mailing list before investing a lot of time, right?)
- Only committers are able to commit changes to the NuttX GIT repositories, but all members of the community are encouraged to participate in reviews and discussions. Committers are encouraged to weigh input from the larger community. If you are not (yet) a committer, being actively involved in the project and demonstrating good will and good judgment, can go a long way toward becoming a committer!
Now we will go into detail on each step:
Obtain a Copy of the Source Code
You can choose to make a GIT clone and develop your changes against it. Alternatively, you can obtain a versioned release tarball. (You can optionally import the contents of a release into whichever SCM / VCS software you prefer.)
Keep in mind that how you submit changes will depend upon what kind of local environment you have created to manage your NuttX-based development:
- If you are using a versioned NuttX release, which comes with no SCM information, you will probably need to send patches.
- If you are using a clone or fork of the GIT repositories, then you might want to send Pull Requests (PRs) to submit changes, but
- If you are using a versioned release branch in the GIT repositories, then you will have to do things a little differently.
Obtaining and Setting Up a Versioned Release
REVISIT: How to obtain the tarball; how to un-tar it; what steps (if any) should be taken before beginning work on customizations.
Obtaining and Setting Up a GIT Clone or Fork
REVISIT: Give detailed instructions on how to clone the repositories, and what steps to take before beginning work on customizations. (I recommend immediately creating a branch with 'git checkout -b <branch name>' .) What is the purpose of this branch? Aren't these "Obtaining" paragraphs redundant? Isn't that addressed above? Some kind of reorganization is needed, either eliminating some redundant paragraphs or moving information from above to here.
Cloning the Repositories: Clones of repositories are useful for creating patches that you can send to the Apache project for inclusion.
These are the steps to clone the repositories:
git clone https://github.com/apache/incubator-nuttx nuttx git clone https://github.com/apache/incubator-nuttx-apps apps
Create a separate branch for your work, this will facilitate creating patches and re-synchronizing with upstream.
git checkout master # Switch to master if not already there. git checkout -b <branch_name> # Create and switch to the branch.
Forking the Repositories: A fork of the repository is a clone of the origin repository that exists in your remote GitHub user area. The "fork & pull" model lets anyone fork an existing repository and push changes to their personal fork without requiring access be granted to the source repository. The changes must then be pulled into the source repository by the project maintainer. This model reduces the amount of friction for new contributors and is popular with open source projects because it allows people to work independently without upfront coordination.
The forked repository is mostly static. It exists in order to allow you to publish work for code review purposes. You don't do active development in your forked repository as would do with a cloned repository. Unlike the cloned repository, the fork does not exist on your computer, it exists on GitHub's server in the cloud. Instead, you would do your development on a clone of the fork of the repository.
To create a fork of your repository, follow these steps (from https://help.github.com/en/github/getting-started-with-github/fork-a-repo):
- Create the fork by visiting the https://github.com/apache/incubator-nuttx/. In the upper right corner, there is a button labeled Fork. Follow the instructions and this will create a repository in your personal area at https://github.com/<username>/incubator-nuttx.
- Create a local clone of your fork. On your local PC execute
git clone <URL>
to create a clone of your fork on your local PC. You can find the exact URL for that command by visiting https://github.com/<username>/incubator-nuttx and clicking on the Clone or download button.
It may be useful to define a remote name for the original repository. You will need this later when you re-synchronize your clone:
Check the list of remotes that you have already:
git remote -v
At this point only origin should exist:
origin https://github.com/<username>/incubator-nuttx.git (fetch) origin https://github.com/<username>/incubator-nuttx.git (push)
Add the NuttX remote:
git remote add upstream https://github.com/apache/incubator-nuttx.git git remote -v
Now a new remote, named upstream, should show up:
origin https://github.com/<username>/incubator-nuttx.git (fetch) origin https://github.com/<username>/incubator-nuttx.git (push) upstream https://github.com/apache/incubator-nuttx.git (fetch) upstream https://github.com/apache/incubator-nuttx.git (push)
Similar to the clone workflow, create a separate branch for your work.
git checkout master # Switch to master if not already there. git checkout -b <branch_name> # Create and switch to the branch. git push -u origin <branch_name> # Push the newly created branch to the fork # and set it to track the remote branch.
You can check the list of branches you have and what remote branches they are tracking with:
git branch -vv
This will output something similar to:
master <commit_sha> [origin/master] <commit_msg> <branch_name> <commit_sha> [origin/<branch_name>] <commit_msg>
Committers Only. It is especially important that NuttX committers use forks rather than clones for code modification. Committers have write privileges directly into NuttX repositories and so also bear the responsibility of being especially careful not to accidentally make unauthorized modifications to the repositories. If committers use GIT clones of the NuttX repositories, then there is the constant risk of an error that inadvertently modifies the upstream repositories. When the committer functions as a contributor, this responsibility is best supported by using a safe fork rather than an uncontrolled clone.
Develop Your Changes
This document describes the workflow of how to get changes upstreamed into the official NuttX repositories. The technical details of how to develop changes are beyond the scope of this document. However, we will take this opportunity to remind you to:
- Be mindful of the Criteria For Acceptance, and
- Feel free to communicate with the dev@nuttx.apache.org mailing list as often as you need to for guidance.
Submit Your Changes For Review
At this point, you have fixed a bug, implemented a new feature, or made some other improvement to your local copy of NuttX. To get your changes into the official NuttX repositories, it helps tremendously if you can package them into a format that conveys the changes accurately.
As stated earlier, how you submit changes will depend upon what kind of local environment you have created to manage your NuttX-based development:
- If you are using a versioned NuttX release, which comes with no SCM information, you will probably need to send patches. See If Working From a Versioned NuttX Release below.
- If you are using a clone or fork of the GIT repositories, then you have several options, including Pull Requests (PRs). See If Working From a Clone or Fork of the GIT Repositories below.
- If you are using a versioned release branch in the GIT repositories, then you will have to do things a little differently. See If Working Using a Versioned Release Branch below.
If Working From a Versioned NuttX Release
REVISIT: Describe in detail, how to make the relevant patch(es) when working with a versioned NuttX release tarball with no SCM information.
If Working From a Clone or Fork of the GIT Repositories
When working from a clone or fork of the GIT repositories, you can submit your changes by one of the following methods:
- Generating a Patch from a Clone
- Generating a GitHub Pull-Request from a Fork
- Generating a Textual GIT Pull-Request from a Clone
Before Submitting Your Changes
You need to ensure that the changes you are going to submit do not conflict with the repository and are ready for a clean merge.
This is achieved by rebasing your commits on top of the current upstream branch.
git fetch upstream # First get the latest changes. git rebase upstream/<branch> # Rebase on top of upstream's branch, could be "master". ... # Fix any conflicts then run git rebase --continue git push --force-with-lease origin <branch> # Push your changes to your feature branch. # A force push is necessary as the history just changed.
Generating a Patch from a Clone
When you are done developing and you are satisfied with your work, you can create a patch using this GIT command:
git format-patch master --stdout > patch.txt
The generated patch set should be sent to dev@nuttx.apache.org. To make sure that your patch email is noticed, please start the subject line with [PATCH]
and a brief description. It is best to send the patch as an attachment with a .txt
extension, as that ensures it gets the correct MIME type and isn't blocked by the mailing list's spam filter.
Your patch will be reviewed by the community and incorporated when everything is okay. If a change is required, address that and provide another patch.
Getting to this point your patch has been accepted and pushed to the main repository.
Now, you need to re-synchronize your local clone. It's important to note that committers can perform some minor changes on your behalf, so always pull the latest changes.
Here are the GIT commands that will help you do that:
git checkout master # Switch to master branch git branch -D <branch_name> # Since the branch has commits that weren't merged, # GIT will complain if you try to delete the branch with -d # Thus, force it with -D git pull origin master # Fetch the latest changes from remote # and merge them to your local repository.
Generating a GitHub Pull-Request from a Fork
If you are working on a fork, you have the option of creating a Pull-Request (PR) instead. To create a pull request on a fork, do the following steps (from https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/creating-a-pull-request-from-a-fork):
- Make your changes on a branch and commit the change to your local clone of the fork using
git commit
- Push your changed code to your private fork:
git push origin <branch_name>
- Then visit your fork at http://github.com/<username>/incubator-nuttx
- To the right of the Branch menu, click New pull request and follow the instructions.
Your changes will now show up as Pull Requests in the original NuttX repository at https://github.com/apache/incubator-nuttx
After your Pull-Request has been accepted, you will need to synchronize your fork with the upstream. You can do this with the following GIT commands:
git checkout master # Switch to master branch git branch -D <branch_name> # Delete the local branch git push -d origin <branch_name> # Delete the remote branch git fetch upstream # Get latest changes from upstream git merge upstream/master # Merge changes with the local master git push origin master # Synchronize the private fork
Often, a fork will diverge too much from the upstream repository to keep in synchronization properly. In this case, feel free to create a new private fork, removing the old forked repository first, of course.
It is a common practice to delete a fork after the pull-request has been closed. GitHub even supports a special shortcut to delete your fork. After the your PR has been closed, go to the https://github.com/apache/incubator-nuttx/pull/NN page (where NN is the PR number that was assigned to your pull request). You will see this message on that page:
Pull request closed
If you wish, you can delete this fork of apache/incubator-nuttx.
Simply click the "Fork Settings" button and confirm that your really want to delete your fork. This applies only if your PR was submitted from the master branch. The message and button label will be different if you submitted the PR from a branch:
Multiple Branches. As a general rule it is important not to mix functionally unrelated code into the same pull-request. Rather, you should consider using multiple branches. If the current branch is tied up waiting for a PR to be closed and you want to submit an unrelated PR, just create a second branch for the second PR. When the PR has been merged, you will be provided an option at https://github.com/apache/incubator-nuttx/pull/NN to delete the branch, instead of deleting the whole fork.
Generating a Textual GIT Pull-Request from a Clone
A textual GIT pull-request is similar to a GIT patch, except that it provides additional context that makes it easier to integrate the change with the upstream repository.
REVISIT: Describe how to make a textual GIT pull request (git request-pull) and send to dev@nuttx.apache.org.
If Working Using a Versioned Release Branch
REVISIT: Describe what must be done differently from the above
Community/Committer Review
Criteria For Acceptance
This section defines what sorts of things committers will examine and verify before allowing changes into NuttX.
This is documented here for three reasons:
- To make it easier for NuttX contributors to get their changes accepted.
- As a checklist of things for NuttX committers to look for when reviewing a potential change.
- As a specification of checks that can be automated over time, eventually saving everyone a lot of work!
First, there are the universal requirements that apply to all parts of NuttX:
- INVIOLABLES, the overarching principles of this project
- NuttX C Coding Standard
- nx-style
- Clean architecture
- POSIX compliance
- Successful build
All new files contributed to the project must begin with an Apache Source File License Header: https://www.apache.org/legal/src-headers.html
In addition to the universal requirements, various parts of the NuttX system have their own specific needs:
- The build system
- Tools
- Architectural support
- Board support
- Scheduler
- System calls
- Libraries
REVISIT: This section is incomplete! Please leave the bulletpoints above as-is, and develop the text in new subheadings below.
Reference For Committers
This section is a guide and reference for committers regarding how to carry out all the steps to process a proposed change from start to finish.
"Start" means that someone contributes a potential change via:
- A GitHub Pull Request (PR), or
- A textual pull request generated with 'git request-pull' and delivered by email, or
- A patch delivered by email.
"Finish" can mean one of:
- Applying the change to master, or
- Requesting additional work from the contributor (i.e., fixing compile errors/warnings, logic errors found in the contributed code, etc.), or
- Rejecting the change because it doesn't meet the needs of the project (e.g., violates the INVIOLABLES or would take the project in a direction that doesn't fit the project mission).
Even veteran NuttX developers and committers are encouraged to read this section and refer back to it, to make their work easier and also to avoid mistakes that might mess up the upstream repositories.
How to Review Changes
Changes should be reviewed to ensure they meet the Criteria For Acceptance described above.
REVISIT: How much "reviewing" is a committer expected to do before deciding that a change should be applied (or sent back for additional work, or rejected outright).
REVISIT: What steps should a reviewer actually take? (I.e., should they bring the change to their local computer and try to build a configuration that includes the change?)
How to Process Changes
Committers who use GitHub can benefit from various conveniences and collaboration tools that GitHub offers as part of the platform.
However, we recognize that some committers cannot or will not use GitHub for various reasons. Therefore, we support both possibilities: GitHub and Plain GIT (Non-GitHub).
GitHub
This section explains to committers how to process changes using GitHub.
GitHub Pull Request (PR)
REVISIT: How to process changes that arrive via GitHub Pull Request.
Emailed Textual Pull Request
REVISIT: How to convert these to a GitHub Pull Request (and then proceed from above).
Emailed Textual Patch
REVISIT: How to convert these to a GitHub Pull Request (and then proceed from above).
Plain GIT (Non-GitHub)
This section explains to committers how to process changes using plain GIT (without GitHub).
GitHub Pull Request (PR)
REVISIT: Is there a way for someone who cannot access GitHub to get access to a change?
Emailed Textual Pull Request
REVISIT: How to process these SAFELY using plain GIT (without messing up the upstream repositories).
Emailed Textual Patch
REVISIT: How to process these SAFELY using plain GIT (without messing up the upstream repositories).