WORK IN PROGRESS

Overview

Purpose

The purpose is simple: We'd like to help everyone participate in the future of NuttX by providing clear and open documentation on how to contribute.

This document can be declared a success if newbies and veteran NuttX developers alike find it valuable as both a how-to and a reference. We hope you'll find it easy to read and that it eliminates the need to hunt, memorize, or guess about parts of the workflow.

Enthusiastic Contributors Welcome!

NuttX is a free open-source project. If you'd like to participate, whether it's to enhance documentation (even this workflow document) or dive into the nitty gritty of some low-level drivers, please read on!

Also, join the conversation at our dev mailing list by emailing dev-subscribe@nuttx.apache.org. The mailing list is open to the public and archived. You can browse older messages at https://lists.apache.org/list.html?dev@nuttx.apache.org.

Why Contribute to NuttX?

Have you fixed a bug or implemented a new feature in your copy of NuttX? If so, contributing your improvements "upstream," which means getting them included in the official NuttX code, may be in your best interest because it means that you won't have to re-fix, re-implement, or port your changes over when we release a newer version of NuttX. It also increases the likelihood of your code getting more exposure and testing, and with that, the possibility of other people contributing additional improvements to your code, which will benefit you "for free."

Who Are The Players?

• Contributor: Anyone who wants to participate in NuttX by contributing improvements.
• Committer: A person who can commit improvements directly to the codebase.
• PMC: Project Management Committee. (Currently, NuttX is a Podling of the Apache Incubator, so currently this is called the PPMC.)
• Upstream: In this document, "upstream" refers to the NuttX project.
• Downstream: In this document, "downstream" means anyone who packages, customizes, and/or uses NuttX in their own projects.

Where is The Code?

Sources of NuttX Code

NuttX source code is available in two forms:

• As a clone or fork from a NuttX GIT repository, or
• As a versioned release in the form of a GZIP'ed tarball

The versioned release tarballs contain no SCM information.  They are intended for use with whatever SCM you select in your custom environment.

Versioned GIT Branches

The code on which versioned releases are based is also available within the NuttX GIT repository clones or forks.  Historical releases are identified with tags.  For example, the tag nuttx-8.2 identifies the nuttx-8.2 release.  A versioned branch could be created with the git command:

git checkout -b <branch-name> nuttx-8.2

The code that you will find on the branch will differ from the the code in the versioned tarballs in two ways:

• It will contain the complete GIT history leading up to the versionsed release, and
• It will not contain the .version file.  That version file exists only in the versioned tarball releases.

REVISIT: We do not yet know the form of releases for future Apache releases.  These will most likely already be on a branch and will include the .version file on that branch. (Note: The release process should be discussed in a different section, or even a completely separate document.) 

NuttX GIT Repositories

The Apache NuttX project uses Git SCM as its version control system.

There are two primary repositories:

• nuttx - The real time operating system itself.
• apps - Optional applications for the operating system. You can use this repository or make your own custom one.

The main "Single Source of Truth" repositories are hosted by the ASF:

• https://gitbox.apache.org/repos/asf?p=incubator-nuttx.git
• https://gitbox.apache.org/repos/asf?p=incubator-nuttx-apps.git

These are synchronized with repositories hosted at GitHub:

• https://github.com/apache/incubator-nuttx/
• https://github.com/apache/incubator-nuttx-apps/

Released Code Tarballs

REVISIT:  We do not know the form or location of future Apache NuttX releases.

Apache project releases are found at https://projects.apache.org/releases.html or https://archive.apache.org/dist/.  However, podling releases are handled differently.  They are released using the Github release system:

• https://github.com/apache/incubator-nuttx/releases
• https://github.com/apache/incubator-nuttx-apps/releases

REVISIT: The above releases are not the official NuttX releases.  They do not include the .version file.  I am not sure how those were generated but they are incorrect.

Perhaps I am confused.  Perhaps the releases will be at https://archive.apache.org/dist/incubator/incubating-nuttx/ which does not yet exist.

REVISIT: This "Overview" section makes no mention of the buildroot, tools, and uclibc repositories. We do not yet know how/where those will be hosted.

The Workflow

All contributors, whether seasoned NuttX committers or new contributors, follow the same steps to submit a proposed change.

First, we will outline the basic steps. Afterwards, we will go into detail on each step.

All Contributors:

• Optional: Before you begin work, discuss your proposed change on the dev@nuttx.apache.org mailing list. Someone else may already be working on a similar change and perhaps you could collaborate and split the effort. Also, a discussion could help you find out whether your changes are likely to be accepted, and input from other community members might provide some valuable insights.
• Review the section "Criteria For Acceptance" in this document.
• Obtain a copy of the source code, if you don't already have it.
• Develop your changes.
• Submit your changes.

Community/Committer Review:

• At this point, the community, especially committers, should review your proposed changes to ensure that the project's criteria are met. These are spelled out in the section "Criteria For Acceptance."
• If any issues are found, you may be asked to correct them and re-submit your changes. Don't be discouraged if this happens; it's a normal part of the open source process. Committers may opt to go ahead and correct minor issues (such as coding style, etc.) instead of sending them back to you.
• Once the review has taken place and the project's criteria are met, your changes are ready to be committed to the official repository. 

Notes:

• No one is obligated to review a change. Generally, committers will review changes that interest them or affect areas they care about. Fortunately, the NuttX community is friendly and has a large following, so chances are pretty good that your changes will be reviewed. (You did discuss your ideas on the mailing list before investing a lot of time, right?)
• Only committers are able to commit changes to the NuttX GIT repositories, but all members of the community are encouraged to participate in reviews and discussions. Committers are encouraged to weigh input from the larger community. If you are not (yet) a committer, being actively involved in the project and demonstrating good will and good judgment, can go a long way toward becoming a committer!

Now we will go into detail on each step:

Obtain a Copy of the Source Code

You can choose to make a GIT clone and develop your changes against it. Alternatively, you can obtain a versioned release tarball. (You can optionally import the contents of a release into whichever SCM / VCS software you prefer.)

Keep in mind that how you submit changes will depend upon what kind of local environment you have created to manage your NuttX-based development:

• If you are using a versioned NuttX release, which comes with no SCM information, you will probably need to send patches.
• If you are using a clone or fork of the GIT repositories, then you might want to send Pull Requests (PRs) to submit changes, but
• If you are using a versioned release branch in the GIT repositories, then you will have to do things a little differently.

Obtaining and Setting Up a Versioned Release

REVISIT: How to obtain the tarball; how to un-tar it; what steps (if any) should be taken before beginning work on customizations.

Obtaining and Setting Up a GIT Clone or Fork

REVISIT: Give detailed instructions on how to clone the repositories, and what steps to take before beginning work on customizations. (I recommend immediately creating a branch with 'git checkout -b <branch name>' .) What is the purpose of this branch?  Aren't these "Obtaining" paragraphs redundant?  Isn't that addressed above?  Some kind of reorganization is needed, either eliminating some redundant paragraphs or moving information from above to here.

Cloning the Repositories:  Clones of repositories are useful for creating patches that you can send to the Apache project for inclusion.
These are the steps to clone the repositories:

git clone https://github.com/apache/incubator-nuttx nuttx
git clone https://github.com/apache/incubator-nuttx-apps apps

Create a separate branch for your work, this will facilitate creating patches and re-synchronizing with upstream.

git checkout master           # Switch to master if not already there.
git checkout -b <branch_name> # Create and switch to the branch.

Forking the Repositories:  A fork of the repository is a clone of the origin repository that exists in your remote GitHub user area.  The "fork & pull" model lets anyone fork an existing repository and push changes to their personal fork without requiring access be granted to the source repository.  The changes must then be pulled into the source repository by the project maintainer. This model reduces the amount of friction for new contributors and is popular with open source projects because it allows people to work independently without upfront coordination.

The forked repository is mostly static.  It exists in order to allow you to publish work for code review purposes. You don't do active development in your forked repository as would do with a cloned repository.  Unlike the cloned repository, the fork does not exist  on your computer, it exists on GitHub's server in the cloud.  Instead, you would do your development on a clone of the fork of the repository.

To create a fork of your repository, follow these steps (from https://help.github.com/en/github/getting-started-with-github/fork-a-repo):

• Create the fork by visiting the https://github.com/apache/incubator-nuttx/.  In the upper right corner, there is a button labeled Fork.  Follow the instructions and this will create a repository in your personal area at https://github.com/<username>/incubator-nuttx.
• Create a local clone of your fork.  On your local PC execute git clone <URL> to create a clone of your fork on your local PC.  You can find the exact URL for that command by visiting https://github.com/<username>/incubator-nuttx and clicking on the Clone or download button.

It may be useful to define a remote name for the original repository.  You will need this later when you re-synchronize your clone:

• Check the list of remotes that you have already:

  git remote -v

  At this point only origin should exist:

  origin https://github.com/<username>/incubator-nuttx.git (fetch)
  origin https://github.com/<username>/incubator-nuttx.git (push)

• Add the NuttX remote:

  git remote add upstream https://github.com/apache/incubator-nuttx.git
  git remote -v

  Now a new remote, named upstream, should show up:

  origin https://github.com/<username>/incubator-nuttx.git (fetch)
  origin https://github.com/<username>/incubator-nuttx.git (push)
  upstream https://github.com/apache/incubator-nuttx.git (fetch)
  upstream https://github.com/apache/incubator-nuttx.git (push)

Similar to the clone workflow, create a separate branch for your work.

git checkout master              # Switch to master if not already there.
git checkout -b <branch_name>    # Create and switch to the branch.
git push -u origin <branch_name> # Push the newly created branch to the fork
                                 # and set it to track the remote branch.

You can check the list of branches you have and what remote branches they are tracking with:

git branch -vv

This will output something similar to:

master        <commit_sha>  [origin/master] <commit_msg>
<branch_name> <commit_sha>  [origin/<branch_name>] <commit_msg>

Committers Only.  It is especially important that NuttX committers use forks rather than clones for code modification.  Committers have write privileges directly into NuttX repositories and so also bear the responsibility of being especially careful not to accidentally make unauthorized modifications to the repositories. If committers use GIT clones of the NuttX repositories, then there is the constant risk of an error that inadvertently modifies the upstream repositories.  When the committer functions as a contributor, this responsibility is best supported by using a safe fork rather than an uncontrolled clone.

Develop Your Changes

This document describes the workflow of how to get changes upstreamed into the official NuttX repositories. The technical details of how to develop changes are beyond the scope of this document. However, we will take this opportunity to remind you to:

• Be mindful of the Criteria For Acceptance, and
• Feel free to communicate with the dev@nuttx.apache.org mailing list as often as you need to for guidance.

Submit Your Changes For Review

At this point, you have fixed a bug, implemented a new feature, or made some other improvement to your local copy of NuttX. To get your changes into the official NuttX repositories, it helps tremendously if you can package them into a format that conveys the changes accurately.

As stated earlier, how you submit changes will depend upon what kind of local environment you have created to manage your NuttX-based development:

• If you are using a versioned NuttX release, which comes with no SCM information, you will probably need to send patches. See If Working From a Versioned NuttX Release below.
• If you are using a clone or fork of the GIT repositories, then you have several options, including Pull Requests (PRs). See If Working From a Clone or Fork of the GIT Repositories below.
• If you are using a versioned release branch in the GIT repositories, then you will have to do things a little differently. See If Working Using a Versioned Release Branch below.

If Working From a Versioned NuttX Release

REVISIT: Describe in detail, how to make the relevant patch(es) when working with a versioned NuttX release tarball with no SCM information.

If Working From a Clone or Fork of the GIT Repositories

When working from a clone or fork of the GIT repositories, you can submit your changes by one of the following methods:

• Generating a Patch from a Clone
• Generating a GitHub Pull-Request from a Fork
• Generating a Textual GIT Pull-Request from a Clone

Before Submitting Your Changes

You need to ensure that the changes you are going to submit do not conflict with the repository and are ready for a clean merge.
This is achieved by rebasing your commits on top of the current upstream branch.

git fetch upstream # First get the latest changes.
git rebase upstream/<branch> # Rebase on top of upstream's branch, could be "master".

... # Fix any conflicts then run git rebase --continue

git push --force-with-lease origin <branch> # Push your changes to your feature branch.
											# A force push is necessary as the history just changed.

As an optional step and desirable if your history contains a series of commits that address the same functional change, you can squash the commits into a single one, this is done with an interactive rebasing.
(For additional information on interactive rebasing, please visit https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History)

git rev-list --count upstream/<branch>..HEAD # Get the number of commits between your branch and upstream's (could be master).
git rebase -i HEAD~N # Where "N" is the number of commits given by the previous command.

... # Your favorite text editor will start and shows the commands for the interactive rebasing.
... # Change the "pick" to "squash" for all the commits you want to squash.
... # For the series of commits to squash, keep the first one with "pick".
... # Once you save and exist your editor will start again to merge all the commit messages.
... # Add a commit message that best describes your change then save and exit.

git push --force-with-lease origin <branch> # Push your changes to your feature branch.
											# A force push is necessary as the history just changed.

Generating a Patch from a Clone

When you are done developing and you are satisfied with your work, you can create a patch using this GIT command:

git format-patch master --stdout > patch.txt

The generated patch set should be sent to dev@nuttx.apache.org. To make sure that your patch email is noticed, please start the subject line with [PATCH] and a brief description. It is best to send the patch as an attachment with a .txt extension, as that ensures it gets the correct MIME type and isn't blocked by the mailing list's spam filter.

Your patch will be reviewed by the community and incorporated when everything is okay. If a change is required, address that and provide another patch.

Getting to this point your patch has been accepted and pushed to the main repository.
Now, you need to re-synchronize your local clone. It's important to note that committers can perform some minor changes on your behalf, so always pull the latest changes.
Here are the GIT commands that will help you do that:

git checkout master         # Switch to master branch
git branch -D <branch_name> # Since the branch has commits that weren't merged,
							# GIT will complain if you try to delete the branch with -d
							# Thus, force it with -D

git pull origin master      # Fetch the latest changes from remote 
                            # and merge them to your local repository.

Generating a GitHub Pull-Request from a Fork

If you are working on a fork, you have the option of creating a Pull-Request (PR) instead.  To create a pull request on a fork, do the following steps (from https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/creating-a-pull-request-from-a-fork):

• Make your changes on a branch and commit the change to your local clone of the fork using git commit
• Push your changed code to your private fork:  git push origin <branch_name>
• Then visit your fork at http://github.com/<username>/incubator-nuttx
• To the right of the Branch menu, click New pull request and follow the instructions.

Your changes will now show up as Pull Requests in the original NuttX repository at https://github.com/apache/incubator-nuttx

After your Pull-Request has been accepted, you will need to synchronize your fork with the upstream.  You can do this with the following GIT commands:

git checkout master              # Switch to master branch
git branch -D <branch_name>      # Delete the local branch
git push -d origin <branch_name> # Delete the remote branch
git fetch upstream               # Get latest changes from upstream
git merge upstream/master        # Merge changes with the local master
git push origin master           # Synchronize the private fork

Often, a fork will diverge too much from the upstream repository to keep in synchronization properly.  In this case, feel free to create a new private fork, removing the old forked repository first, of course.
It is a common practice to delete a fork after the pull-request has been closed.  GitHub even supports a special shortcut to delete your fork.  After the your PR has been closed, go to the https://github.com/apache/incubator-nuttx/pull/NN page (where NN is the PR number that was assigned to your pull request).  You will see this message on that page:

Pull request closed

If you wish, you can delete this fork of apache/incubator-nuttx.

Simply click the "Fork Settings" button and confirm that your really want to delete your fork.  This applies only if your PR was submitted from the master branch.  The message and button label will be different if you submitted the PR from a branch:

Multiple Branches.  As a general rule it is important not to mix functionally unrelated code into the same pull-request.  Rather, you should consider using multiple branches.  If the current branch is tied up waiting for a PR to be closed and you want to submit an unrelated PR, just create a second branch for the second PR.  When the PR has been merged, you will be provided an option at https://github.com/apache/incubator-nuttx/pull/NN to delete the branch, instead of deleting the whole fork.

Generating a Textual GIT Pull-Request from a Clone

A textual GIT pull-request is similar to a GIT patch, except that it provides additional context that makes it easier to integrate the change with the upstream repository.

REVISIT: Describe how to make a textual GIT pull request (git request-pull) and send to dev@nuttx.apache.org.

If Working Using a Versioned Release Branch

REVISIT: Describe what must be done differently from the above

Community/Committer Review

Criteria For Acceptance

This section defines what sorts of things committers will examine and verify before allowing changes into NuttX.

This is documented here for three reasons:

• To make it easier for NuttX contributors to get their changes accepted.
• As a checklist of things for NuttX committers to look for when reviewing a potential change.
• As a specification of checks that can be automated over time, eventually saving everyone a lot of work!

First, there are the universal requirements that apply to all parts of NuttX:

• INVIOLABLES, the overarching principles of this project
• NuttX C Coding Standard
• nx-style
• Clean architecture
• POSIX compliance
• Successful build

All new files contributed to the project must begin with an Apache Source File License Header: https://www.apache.org/legal/src-headers.html

In addition to the universal requirements, various parts of the NuttX system have their own specific needs:

• The build system
• Tools
• Architectural support
• Board support
• Scheduler
• System calls
• Libraries

REVISIT: This section is incomplete! Please leave the bulletpoints above as-is, and develop the text in new subheadings below.

Reference For Committers

This section is a guide and reference for committers regarding how to carry out all the steps to process a proposed change from start to finish.

"Start" means that someone contributes a potential change via:

• A GitHub Pull Request (PR), or
• A textual pull request generated with 'git request-pull' and delivered by email, or
• A patch delivered by email.

"Finish" can mean one of:

• Applying the change to master, or
• Requesting additional work from the contributor (i.e., fixing compile errors/warnings, logic errors found in the contributed code, etc.), or
• Rejecting the change because it doesn't meet the needs of the project (e.g., violates the INVIOLABLES or would take the project in a direction that doesn't fit the project mission).

Even veteran NuttX developers and committers are encouraged to read this section and refer back to it, to make their work easier and also to avoid mistakes that might mess up the upstream repositories.

How to Review Changes

Changes should be reviewed to ensure they meet the Criteria For Acceptance described above.

REVISIT: How much "reviewing" is a committer expected to do before deciding that a change should be applied (or sent back for additional work, or rejected outright).

REVISIT: What steps should a reviewer actually take? (I.e., should they bring the change to their local computer and try to build a configuration that includes the change?)

How to Process Changes

Committers who use GitHub can benefit from various conveniences and collaboration tools that GitHub offers as part of the platform.

However, we recognize that some committers cannot or will not use GitHub for various reasons. Therefore, we support both possibilities: GitHub and Plain GIT (Non-GitHub).

GitHub

This section explains to committers how to process changes using GitHub.

GitHub Pull Request (PR)

REVISIT: How to process changes that arrive via GitHub Pull Request.

Emailed Textual Pull Request

REVISIT: How to convert these to a GitHub Pull Request (and then proceed from above).

Emailed Textual Patch

REVISIT: How to convert these to a GitHub Pull Request (and then proceed from above).

Plain GIT (Non-GitHub)

This section explains to committers how to process changes using plain GIT (without GitHub).

GitHub Pull Request (PR)

REVISIT: Is there a way for someone who cannot access GitHub to get access to a change?

Emailed Textual Pull Request

REVISIT: How to process these SAFELY using plain GIT (without messing up the upstream repositories).

Emailed Textual Patch

REVISIT: How to process these SAFELY using plain GIT (without messing up the upstream repositories).