You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 54 Next »

WORK IN PROGRESS


Email Context

Nathan sent this email which is a reasonable outline.  So I have it here for a reference to work toward, I will be making some changes where I think are appropriate.  At the end of the day this probably should go into a CONTRIBUTING file on GitHub.

At this point, much has been written in this and other threads.

Now I recommend that we should have a Confluence page where the information
can be edited to make it:

* Plain English
* Flow well
* Readable
* Coherent
* Specific
* Not assume knowledge
* In one easy to find place

I say "I recommend that we should" instead of just going ahead and creating
that Confluence page because I want to ensure that this happens with
community agreement.

Rationale for creating the document: I don't want or expect anyone to
memorize our workflow, nor to go digging all over the place to piece
together what it is. I want it documented clearly so that veteran
contributors and total newbies alike will be able to understand exactly
what to do. I also want this documented for purpose of on-boarding future
committers and so that committers will know what is expected of them
whenever they commit. And furthermore I want it in a clear document that we
can officially vote on to make it our "blessed" workflow, until such time
as we decide to make changes and vote to update the "blessed" process.

The document should flow in a manner that makes information easy to find. I
suggest to organize it as follows:

1. Overview. This section defines "WHAT" the steps of the workflow are.
This is a simple bulleted or numbered list. No rationale, no git jargon, no
DevOps nonsense, just clear plain English. Where is the code. What basic
steps take place to apply changes to it. Greg literally wrote this list a
few emails ago and it contains 80% of what should be in that list. It needs
minimal improvement.

2. How To Submit Changes For Review. This section defines the "HOW" of the
workflow for ANYONE who wants to get a change into NuttX, whether committer
or not. Committers do not get to bypass this process. This section should
document: How to obtain a copy of the code. What steps to take before
beginning work on a change. Once you've made your change and want to
contribute it, what steps to take to turn it into a PR or patch. How to
submit the PR or patch to us? We should NOT assume knowledge. If a step
requires using git, then give the exact git command followed by an
explanation of every element of that command, so that anyone who knows how
to enter a command in a terminal with zero prior knowledge of git will be
able to understand exactly how to issue that git command and exactly what
it will do. I want to make it straightforward and easy for a HARDWARE
engineer to be able to submit changes.

3. Criteria For Acceptance. This section defines what sorts of things
committers will examine and verify before allowing changes into NuttX.
First, the universal requirements that apply to all parts of NuttX. This
includes INVIOLABLES, coding standard, rules that govern clean
architecture, POSIX compliance, not breaking the build, etc. The word
INVIOLABLES should be a link to that file. After the universal
requirements, there should be requirements by area. Requirements for
boards. Requirements for drivers. Requirements for the scheduler.
Requirements for architectural support. Etc. All of the checks in this
section can be performed manually by a committer for now. This section
should be documented clearly and specifically enough that it can directly
be used as the specification of automated checks to implement in a CI
system.

4. Reference For Committers. This section explains to committers how to
carry out all the steps to process a proposed change from start to finish.
"Start" means a patch or a PR arrived. If a patch, how to convert it into a
PR. "Finish" can mean either applying the change to master, or reject the
change / send it back to the submitter for additional work. And all steps
in between. This section (item #4) is a continuation of item #2. Item #2
explained how anyone submits a proposed change for review. Now we explain
what happens next. Like item #2, we should NOT assume knowledge. If a step
requires using git, then give the exact git command followed by an
explanation of every element of that command. This is not excessive. This
will help on-boarding of new committers as well as help veteran committers
avoid mistakes.

5. Rationale. This section explains the "WHY" behind all of the above. So
that people won't cut an inch off each end of a ham without knowing why.

Cheers,
Nathan

From My Email  (will drop this after I fill in the rest)

  • Submit a PR on GitHub against master if it is approved by one commiter (that did not propose it) it can be merged.  The approval is done via the GitHub approval system.

  • A commiter may create a PR on behalf of a patch submitted to the mailing list.

  • Commiters can ask for others to review or approve.  But at the end of the day they are the ones who approve and merge. 

Overview

Purpose

The purpose is simple: We'd like to help everyone participate in the future of NuttX by providing clear and open documentation on how to contribute.

This document can be declared a success if newbies and veteran NuttX developers alike find it valuable as both a how-to and a reference. We hope you'll find it easy to read and that it eliminates the need to hunt, memorize, or guess about parts of the workflow.

Enthusiastic Contributors Welcome!

NuttX is a free open-source project. If you'd like to participate, whether it's to enhance documentation (even this workflow document) or dive into the nitty gritty of some low-level drivers, please read on!

Also, join the conversation at our dev mailing list by emailing dev-subscribe@nuttx.apache.org. The mailing list is open to the public and archived. You can browse older messages at https://lists.apache.org/list.html?dev@nuttx.apache.org.

Why Contribute to NuttX?

Have you fixed a bug or implemented a new feature in your copy of NuttX? If so, contributing your improvements "upstream," which means getting them included in the official NuttX code, may be in your best interest because it means that you won't have to re-fix, re-implement, or port your changes over when we release a newer version of NuttX. It also increases the likelihood of your code getting more exposure and testing, and with that, the possibility of other people contributing additional improvements to your code, which will benefit you "for free."

Who Are The Players?

  • Contributor: Anyone who wants to participate in NuttX by contributing improvements.
  • Committer: A person who can commit improvements directly to the codebase.
  • PMC: Project Management Committee. (Currently, NuttX is a Podling of the Apache Incubator, so currently this is called the PPMC.)
  • Upstream: In this document, "upstream" refers to the NuttX project.
  • Downstream: In this document, "downstream" means anyone who packages, customizes, and/or uses NuttX in their own projects.

Where is The Code?

Sources of NuttX Code

NuttX source code is available in two forms:

  • As a clone or fork from a NuttX GIT repository, or
  • As a versioned release in the form of a GZIP'ed tarball

The versioned release tarballs contain no SCM information.  They are intended for use with whatever SCM you select in your custom environment.

Versioned GIT Branches

The code on which versioned releases are based is also available within the NuttX GIT repository clones or forks.  Historical releases are identified with tags.  For example, the tag nuttx-8.2 identifies the nuttx-8.2 release.  A versioned branch could be created with the git command:

git checkout -b <branch-name> nuttx-8.2

The code that you will find on the branch will differ from the the code in the versioned tarballs in two ways:

  • It will contain the complete GIT history leading up to the versionsed release, and
  • It will not contain the .version file.  That version file exists only in the versioned tarball releases.

REVISIT: We do not yet know the form of releases for future Apache releases.  These will most likely already be on a branch and will include the .version file on that branch. (Note: The release process should be discussed in a different section, or even a completely separate document.) 

NuttX GIT Repositories

The Apache NuttX project uses Git SCM as its version control system.

There are two primary repositories:

  • nuttx - The real time operating system itself.
  • apps - Optional applications for the operating system. You can use this repository or make your own custom one.

The main "Single Source of Truth" repositories are hosted by the ASF:

These are synchronized with repositories hosted at GitHub:

Cloning or Forking the Repositories

Cloning the Repositories:  These are the steps to clone the source code from repository:

These will create copies, clones, of the Apache NuttX repositories on your local computer.

Enter inside "nuttx" directory and, if you like, create a branch where you will work:

  • cd nuttx
  • git checkout -b mybranch origin/dev

Make your modification and commit them as described later in this document.

Clones of repositories are useful for creating patches that you can send to the Apache project for inclusion.  A better workflow, however, would use pull requests and to use pull requests you will need to use a fork instead of a clone.

Forking the Repositories:  A fork of the repository is a clone of the origin repository that exists in your remote GitHub user area.  The "fork & pull" model lets anyone fork an existing repository and push changes to their personal fork without requiring access be granted to the source repository.  The changes must then be pulled into the source repository by the project maintainer. This model reduces the amount of friction for new contributors and is popular with open source projects because it allows people to work independently without upfront coordination.

The forked repository is mostly static.  It exists in order to allow you to publish work for code review purposes. You don't do active development in your forked repository as would do with a cloned repository.  Unlike the cloned respository, the fork does not exist  on your computer, it exists on GitHub's server in the cloud.  Instead, you would do your development on a clone of the fork of the repository.

To create a fork of your repository, follow these steps (from https://help.github.com/en/github/getting-started-with-github/fork-a-repo):

  • Create the fork by visiting the https://github.com/apache/incubator-nuttx/.  In the upper right corner, there is a button labeled Fork.  Follow the instructions and this will create a repository in your personal area at https://github.com/<username>/incubator-nuttx.
  • Create a local clone of your fork.  On your local PC execute 'git clone <URL>' to create a clone of your fork on your local PC.  You can find the exact URL for that command by visiting https://github.com/<username>/incubator-nuttx and clicking on the Clone or download button.

It may be useful to define a remote name for the original repository.  You will need this later when you re-synchronize your clone:

  • Check the list of remotes that you have already:

git remote -v
origin https://github.com/<username>/incubator-nuttx.git (fetch)
origin https://github.com/<username>/incubator-nuttx.git (push)

  • Add the NuttX remote:

git remote add upstream https://github.com/apache/incubator-nuttx.git

git remote -v
origin https://github.com/<username>/incubator-nuttx.git (fetch)
origin https://github.com/<username>/incubator-nuttx.git (push)
upstream https://github.com/apache/incubator-nuttx.git (fetch)
upstream https://github.com/apache/incubator-nuttx.git (push)

Committers Only.  It is especially important that NuttX committers use forks rather than clones for code modification.  Committers have write privileges directly into NuttX repositories and so also bear the responsibility of being especially careful not to accidentally make unauthorized modifications to the repositories. If committers use GIT clones of the NuttX repositories, then there is the constant risk of an error that inadvertently modifies the upstream repositories.  When the committer functions as a contributor, this responsibility is best supported by using a safe fork rather than an uncontrolled clone.

Released Code Tarballs

REVISIT:  We do not know the form or location of future Apache NuttX releases.

Apache project releases are found at https://projects.apache.org/releases.html or https://archive.apache.org/dist/.  However, podling releases are handled differently.  They are released using the Github release system:

REVISIT: The above releases are not the official NuttX releases.  They do not include the .version file.  I am not sure how those were generated but they are incorrect.

Perhaps I am confused.  Perhaps the releases will be at https://archive.apache.org/dist/incubator/incubating-nuttx/ which does not yet exist.

REVISIT: This "Overview" section makes no mention of the buildroot, tools, and uclibc repositories. We do not yet know how/where those will be hosted.

The Workflow

All contributors, whether seasoned NuttX committers or new contributors, follow the same steps to submit a proposed change.

First, we will outline the basic steps. Afterwards, we will go into detail on each step.

All Contributors:

  • Optional: Before you begin work, discuss your proposed change on the dev@nuttx.apache.org mailing list. Someone else may already be working on a similar change and perhaps you could collaborate and split the effort. Also, a discussion could help you find out whether your changes are likely to be accepted, and input from other community members might provide some valuable insights.
  • Review the section "Criteria For Acceptance" in this document.
  • Obtain a copy of the source code, if you don't already have it.
  • Develop your changes.
  • Submit your changes.

Community/Committer Review:

  • At this point, the community, especially committers, should review your proposed changes to ensure that the project's criteria are met. These are spelled out in the section "Criteria For Acceptance."
  • If any issues are found, you may be asked to correct them and re-submit your changes. Don't be discouraged if this happens; it's a normal part of the open source process. Committers may opt to go ahead and correct minor issues (such as coding style, etc.) instead of sending them back to you.
  • Once the review has taken place and the project's criteria are met, your changes are ready to be committed to the official repository. 

Notes:

  • No one is obligated to review a change. Generally, committers will review changes that interest them or affect areas they care about. Fortunately, the NuttX community is friendly and has a large following, so chances are pretty good that your changes will be reviewed. (You did discuss your ideas on the mailing list before investing a lot of time, right?)
  • Only committers are able to commit changes to the NuttX GIT repositories, but all members of the community are encouraged to participate in reviews and discussions. Committers are encouraged to weigh input from the larger community. If you are not (yet) a committer, being actively involved in the project and demonstrating good will and good judgment, can go a long way toward becoming a committer!

Now we will go into detail on each step:

Obtain a Copy of the Source Code

You can choose to make a GIT clone and develop your changes against it. Alternatively, you can obtain a versioned release tarball. (You can optionally import the contents of a release into whichever SCM / VCS software you prefer.)

Keep in mind that how you submit changes will depend upon what kind of local environment you have created to manage your NuttX-based development:

  • If you are using a versioned NuttX release, which comes with no SCM information, you will probably need to send patches.
  • If you are using a clone or fork of the GIT repositories, then you might want to send Pull Requests (PRs) to submit changes, but
  • If you are using a versioned release branch in the GIT repositories, then you will have to do things a little differently.

Obtaining and Setting Up a Versioned Release

REVISIT: How to obtain the tarball; how to un-tar it; what steps (if any) should be taken before beginning work on customizations.

Obtaining and Setting Up a GIT Clone or Fork

REVISIT: Give detailed instructions on how to clone the repositories, and what steps to take before beginning work on customizations. (I recommend immediately creating a branch with 'git checkout -b <branch name>' .) What is the purpose of this branch?  Aren't these "Obtaining" paragraphs redundant?  Isn't that addressed above?  Some kind of reorganization is needed, either eliminating some redundant paragraphs or moving information from above to here.

Develop Your Changes

This document describes the workflow of how to get changes upstreamed into the official NuttX repositories. The technical details of how to develop changes are beyond the scope of this document. However, we will take this opportunity to remind you to:

  • Be mindful of the Criteria For Acceptance, and
  • Feel free to communicate with the dev@nuttx.apache.org mailing list as often as you need to for guidance.
  • Submit Your Changes
  • At this point, you have fixed a bug, implemented a new feature, or made some other improvement to your local copy of NuttX. To get your changes into the official NuttX repositories, it helps tremendously if you can package them into a format that conveys the changes accurately.

Submit Your Changes For Review

As stated earlier, how you submit changes will depend upon what kind of local environment you have created to manage your NuttX-based development:

  • If you are using a versioned NuttX release, which comes with no SCM information, you will probably need to send patches. See If Working From a Versioned NuttX Release below.
  • If you are using a clone or fork of the GIT repositories, then you have several options, including Pull Requests (PRs). See If Working From a Clone or Fork of the GIT Repositories below.
  • If you are using a versioned release branch in the GIT repositories, then you will have to do things a little differently. See If Working Using a Versioned Release Branch below.

If Working From a Versioned NuttX Release

REVISIT: Describe in detail, how to make the relevant patch(es) when working with a versioned NuttX release tarball with no SCM information.

If Working From a Clone or Fork of the GIT Repositories

When working from a clone or form of the GIT repositories, you can submit your changes by one of the following methods:

  • Generating a Patch from a Clone
  • Generating a GitHub Pull-Request from a Fork
  • Generating a Textual GIT Pull-Request from a Clone

Generating a Patch from a Clone

If you are working from a clone of the repository, you can create a patch using this GIT command:

git format-patch -<n> <sha>

This will create a patches based on the last number of commits <n> starting at the commit <sha>.  Or

git format-patch -<n> HEAD

or which will create a patches starting at the HEAD of the branch.   if you have changes on a separate branch, then you an use rhe following to create the patches.

format-patch <remote>/<branch>

The generated patch set should be sent to dev@nuttx.apache.org. To make sure that your patch email is noticed, please start the subject line with [PATCH] and a brief description. It is best to send the patch as an attachment with a .txt extension, as that ensures it gets the correct MIME type and isn't blocked by the mailing list's spam filter.

Generating a GitHub Pull-Request from a Fork

If you are working on a fork, you have the option of creating a Pull-Request (PR) instead.  To create a pull request on a fork, do the following steps (from https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/creating-a-pull-request-from-a-fork):

  • Make your your changes on a branch (or even on master) and commit the change to your local clone of the fork using 'git commit'
  • Push your changed code to your private fork:  'git push origin <branch>' where <branch> may be master.
  • Then visit your fork at http://github.com/<username>/incubator-nuttx
  • To the right of the Branch menu, click New pull request and follow the instructions.

Your changes will now show up as Pull Requests in the original NuttX repository at https://github.com/apache/incubator-nuttx

After your Pull-Request has been accepted, you will need to synchronize your fork with the upstream.  Instructions for doing this are available here: https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/syncing-a-fork.  Often, a fork will diverge too much from the upstream repository to keep in synchronization properly.  In this case, feel free to create a new private fork, removing the old forked repository first, of course.

It is a common practice to delete a fork after the pull-request has been closed.  GitHub even supports a special shortcut to delete your fork.  After the your PR has been closed, go to the https://github.com/apache/incubator-nuttx/pull/NN page (where NN is the PR number that was assigned to your pull request).  You will see this message on that page:

Pull request closed

If you wish, you can delete this fork of apache/incubator-nuttx.

Simply click the "Fork Settings" button and confirm that your really want to delete your fork.  This applies only if your PR was submitted from the master branch.  The message and button label will be different if you submitted the PR from a branch:

Multiple Branches.  As a general rule it is important not to mix functionally unrelated code into the same pull-request.  Rather, you should consider using multiple branches.  If the current branch is tied up waiting for a PR to be closed and you want to submit an unrelated PR, just create a second branch for the second PR.  When the PR has been merged you will provided at option at https://github.com/apache/incubator-nuttx/pull/NN to delete the branch, instead of deleting the whole fork.

Generating a Textual GIT Pull-Request from a Clone

A textual GIT pull-request is similar to a GIT patch, except that it provides additional context that makes it easier to integrate the change with the upstream repository.

REVISIT: Describe how to make a textual GIT pull request (git request-pull) and send to dev@nuttx.apache.org.

If Working Using a Versioned Release Branch

REVISIT: Describe what must be done differently from the above

Community/Committer Review


Criteria For Acceptance

This section defines what sorts of things committers will examine and verify before allowing changes into NuttX.

This is documented here for three reasons:

  • To make it easier for NuttX contributors to get their changes accepted.
  • As a checklist of things for NuttX committers to look for when reviewing a potential change.
  • As a specification of checks that can be automated over time, eventually saving everyone a lot of work!

First, there are the universal requirements that apply to all parts of NuttX:

All new files contributed to the project must begin with an Apache Source File License Header: https://www.apache.org/legal/src-headers.html

In addition to the universal requirements, various parts of the NuttX system have their own specific needs:

  • The build system
  • Tools
  • Architectural support
  • Board support
  • Scheduler
  • System calls
  • Libraries

REVISIT: This section is incomplete! Please leave the bulletpoints above as-is, and develop the text in new subheadings below.

Reference For Committers

This section is a guide and reference for committers regarding how to carry out all the steps to process a proposed change from start to finish.

"Start" means that someone contributes a potential change via:

  • A GitHub Pull Request (PR), or
  • A textual pull request generated with 'git request-pull' and delivered by email, or
  • A patch delivered by email.

"Finish" can mean one of:

  • Applying the change to master, or
  • Requesting additional work from the contributor (i.e., fixing compile errors/warnings, logic errors found in the contributed code, etc.), or
  • Rejecting the change because it doesn't meet the needs of the project (e.g., violates the INVIOLABLES or would take the project in a direction that doesn't fit the project mission).

Even veteran NuttX developers and committers are encouraged to read this section and refer back to it, to make their work easier and also to avoid mistakes that might mess up the upstream repositories.

How to Review Changes

Changes should be reviewed to ensure they meet the Criteria For Acceptance described above.

REVISIT: How much "reviewing" is a committer expected to do before deciding that a change should be applied (or sent back for additional work, or rejected outright).

REVISIT: What steps should a reviewer actually take? (I.e., should they bring the change to their local computer and try to build a configuration that includes the change?)

How to Process Changes

Committers who use GitHub can benefit from various conveniences and collaboration tools that GitHub offers as part of the platform.

However, we recognize that some committers cannot or will not use GitHub for various reasons. Therefore, we support both possibilities: GitHub and Plain GIT (Non-GitHub).

GitHub

This section explains to committers how to process changes using GitHub.

GitHub Pull Request (PR)

REVISIT: How to process changes that arrive via GitHub Pull Request.

Emailed Textual Pull Request

REVISIT: How to convert these to a GitHub Pull Request (and then proceed from above).

Emailed Textual Patch

REVISIT: How to convert these to a GitHub Pull Request (and then proceed from above).

Plain GIT (Non-GitHub)

This section explains to committers how to process changes using plain GIT (without GitHub).

GitHub Pull Request (PR)

REVISIT: Is there a way for someone who cannot access GitHub to get access to a change?

Emailed Textual Pull Request

REVISIT: How to process these SAFELY using plain GIT (without messing up the upstream repositories).

Emailed Textual Patch

REVISIT: How to process these SAFELY using plain GIT (without messing up the upstream repositories).


  • No labels