Page History

...

• Avoid "integration hell" by provide a quick but thoughtful automated check of new code addition so that the Code Contribution Workflow can be done seamlessly.
  • Definition of quick is difficult, but some hints are:
    • At maximum, a few minutes to rebase local developer work with latest master. Usually a developer get a lot more code than he produces, so is important the rebase to be easy to do, or the developer will prefer to just keep working for a few days, leading to potential "integration hell".
    • At maximum, a couple hours to do minimum set of checks in the CI server. This is needed to not saturate CI service.
    • Less than a day for a full regression including all the tests (nightly tests). This is needed to have a daily stable base.
    • Fail fast approach: the fastest tests should be the first one to be tested, and they should provide feedback right away, stopping any further processing until they are addressed so that CI resources could be freed for testing other changes. Nevertheless, the CI pipeline should be reproducible by local developer environment, allowing the developer to test before submit, so minimizing the developer coding loop.
  • Definition of thoughtful is even more difficult, but some hints are:
    • Check the source code style
    • Check it builds and executes in the main architectures
    • Check it don't break the main functionalities
    • Check that all the new code is covered by the existing or new added tests.
    • Provides useful logs, errors and reports, so that is easy for the developer to fix the problems.
    • The results are consistent (the result of a given commit does not change over time or execution environment)
  • By automated, it would be executed in a CI like ASF Jenkins service. However, the CI platform specific part should be clearly separated from the parts that should be common to the testing in local developer environment. For example, if there is a Jenkins pipeline that is executed, it should use some external script that can be use without a Jenkins environment, and a tool to run the script locally in a similar way to the Jenkins pipeline should be provided.
• Use standardized tools and methods whenever it is possible and practical so that more people can contribute.

...

Current proposal for implementation is to generate a three Jenkins pipeline job, called nuttx.

...

jobs: incubator-nuttx, incubator-nuttx-apps, incubator-nuttx-testing (specially the first one).

The execution will consist on a Jenkins multibranch pipeline defined by the contents of tools/Jenkinsfile file in the root each repository.

Using a multibranch pipeline allows to properly track the master, release and other branches and also the ones created due to pull requests. It also integrates smoothly with new Jenkins interfaces (Blue Ocean) for better/simpler visualization of the pipelines.

Pipeline versioning

As commented, a Jenkins multibranch pipeline defined by the contents of tools/Jenkinsfile file in the root the ci-configuration branch of each repository. But importantly, as demanded by explicit requirement, that file will not contain the logic of the pipeline itself. Instead, it will use a pipeline library defined in the incubator-nuttx-testing repository (see https://jenkins.io/doc/book/pipeline/getting-started/#defining-a-pipeline-in-scm)  repo.

This way, the new changes to the CI logic will not need changes in the main repos, like incubator-nuttx or incubator-nuttx-apps and all the changes will go to incubator-nuttx-testing.

Triggers

The triggers of the pipeline pipelines will be:

• Pull requests master branch (or release/topic branch) of incubator-nuttx repository
• Pull requests master branch (or release/topic branch) of incubator-nuttx-apps repository
• Pull requests master branch (or release/topic branch) of incubator-nuttx-testing repository

WARNING: the special branch in incubator-nuttx-testing repository, called ci-configuration, will have a special functionality as explained before, and must not be used as trigger source. This branch will not trigger any automated job from its pull request. Instead, it should be tested using another Jenkins pipeline job, called nuttx-testing-ci-configuration, that should be triggered manually by Apache NuttX committers with enough privileges. This will allow to test in a more controlled way the changes in the pipeline itselfNOTE: the Jenkins multibranch pipeline jobs will be configured to restrict pull requests that modify the tools/Jenkinsfile to committers, to avoid potential security issues by abuse of not trusted users.

Environment

The jobs in the pipeline will have the following filesystem as part of their environment:

...