Development Process

Summary

Apache CloudStack is a broad and encompassing project covering many areas that require in-depth technical knowledge.  As such, it is not possible for one person to know and/or test all components of CloudStack in depth.  Miscommunications and misunderstandings between developers can lead to poor quality in CloudStack's code and cause community to deliver a unstable release.  Knowledge lost from one release to the next can cause regressions in the current release.  This page details the development process the CloudStack community must follow in order to ensure the community delivers high quality releases.  This page is a must read for all developers and testers.

Process Principles

This process is developed with the following principles.

• Continuous integration is key to a stable code base.
• Automation is the only insurance the community has against regressions in releases.
• Constrain the actual hardware testing to testing CloudStack's physical/virtual orchestration.
• Use simulators to speed up testing of CloudStack's self-service business logic.
• Properly performed code reviews are key to catching problems early.

Development Process

Infrastructure Setup

The development infrastructure is setup as follows.

• Continuous Integration Testing is always running on master and the current release branch.
• Developers can request from Jenkins that automation be run on their branch to make sure the new code and new testcases did not break continuous integration.
• No merges are allowed without a successful run of the automation.

Checkin and Merge Process

The process is very simple.

1. Create a branch for your code or the patch you're helping to submit (Use github if you are not yet a committer)
2. Assign the JIRA issue to yourself, change the status to "In Progress", add the branch information into the issue.
3. When the work is completed, do the following things.
   1. Arrange for a review with two colleagues.
   2. Use Jenkins to ask automation be run on your branch. (Steps below)
4. Once review and automation are completed successfully, merge the branch to the release branch or master. 
5. Once the merge is completed, change the status of the issue to "Resolved."

Review Process

• It is up to the developer to find the right people to review his/her code changes.  
• It is within the reviewer's right to decide whether they are the appropriate reviewer for a code change.  
• If they are not or if the review requires technical expertise from others, the reviewer can either refer the developer to another reviewer or pull in another reviewer.

Automation Process

• Developers uses Jenkins to schedule automation test (BVT) to be performed on their branch.
• If the BVT is not completed successfully, the BVT uploads all logs collected and the testcase that failed.
• Developers can fix their code/tests and repeat until the BVT is successfully completed.
• If a developer cannot figure out the problem from the logs uploaded, they can add more logging to more precisely pinpoint the error and reschedule the BVT on their branch.

Continuous Integration

The BVT will be run on master and the current release branch on a continuous basis.  If the BVT fails, the commits submitted between the last successful BVT run and the failed BVT runs are reverted.  The developers who submitted the commits are notified of the revert and can merge their changes again when they have figured out the problem on their own branch.

Developer Responsibilities

As a code contributor to Apache CloudStack, regardless of if you have committer or contributor status, your responsibilities are as follows:

Acquiring Knowledge

• Understand how to use JIRA to communicate your work with release managers, users, developers, and testers.
• Understand CloudStack's architecture and where you should or should not make changes.
• Understand CloudStack's simulator and how to run it and how to change it to test your code.
• Understand CloudStack's Marvin framework and how to use it to write automated tests.
• Understand CloudStack's code check-in process.
• Understand CloudStack's review process and your role in it as the reviewer and the code submitter.

JIRA

Communicating what issues you're working on and how you're working on it is an important part of participating in the community.  We do that on the mailing list.  However, you should use JIRA to update issue status and add comments with conclusions drawn from mailing list discussions.  This allows release managers, users, and testers to follow the issue better, to understand if the issue is being worked on, to get a better feel of whether an issue will be resolved in time for the release.

Writing Automated Tests

Automated tests ensures that your feature will not be regressed.  Anyone who submits a code change that breaks an existing automated test is responsible for fixing that automated test or revert the code change.