There's a veto outstanding on the introduction of git-flow so this manual is very much work-in-progress. See https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=44303185 for an older version of this page which does not mention git-flow. |
Here is how to kick start feature development:
git clone --recursive https://github.com/<username>/cloudstack.git git flow init git checkout -b <feature-branch>
or if you use git-flow git flow feature start <feature> git checkout <feature-branch>
This fetches remote changes into the git index but not yet into the checkout: git fetch upstream git rebase --ff-only upstream/<feature-branch> git merge upstream/<feature-branch> --no-ffgit config reviewboard.url https://reviews.apache.orgpost-review git push upstream <feature-branch> git checkout develop git pull --rebase git merge --edit --no-ff <feature-branch>
git push upstream develop git branch -d <feature-branch> git flow feature finish <feature>Committers will not be looking at your forked repo by default. Therefore, as a non-committer, you should alert the cloudstack-dev mailing list of your work and periodically request the incremental commits you make be reviewed.
It is also best to work on feature branches rather than working directly on develop, even in your private fork. When the patch is ready for submission push the squashed patch through reviewboard. If the patch is sizeable and touches many projects in the repo, best to break it down into smaller chunks for easier and quicker review by multiple committers. Don't use github's pull request feature, but in the reviewboard patch notes, you can include the upstream you request a committer to pull from to preserve useful history.
Holler for help on the lists if you're stuck and ever need help.
# Set the name of the user for all git instances on the system git config --global user.name "Firstname Lastname" # Set the email-address of the user for all git instances on the system git config --global user.email "your_email@youremail.com" |
Here are some key guidelines for how to make a clean commit. See the linux kernel patch guidelines for more.
It is more important to create clean commits than it is to follow all the branching rules. If your commits are clean, branching/merging mistakes can be resolved by looking at them. The reverse is not true.
Coming from centralized version control like cvs or svn, you may be used to committing you work only once a task is finished. An svn commit goes to the remote repository immediately, so committing work in progress breaks flow for your colleagues that then have to update their working copies. There can also be a race to get your work in first so that you don't have to be the one to sort out the merge.
Leave that mindset behind! Git is truly distributed version control, so you can and should commit without push. In a git workflow, your code-test-code cycle should become
Then when you end a session (when you finish a task or feature, or you reach a milestone in its development), you think about pushing your changes.
git log to remember what commits you didgit rebase -i to clean up your history (optional; see below)git commit --amend to clean up your commit message (see below)git fetch upstream/...; git pull --rebase to replay your changes against the latest upstreamgit push to send your commits upstreamBecause we use feature branches, none of this involves scary merging. Only once a feature is complete does it get merged into the develop branch, and at that point you want to spend some time to check that your changes don't conflict.
Apply, basically, the same principles to commits that apply to source code.
Commits should do one thing and do that thing well.
Either move files, or refactor/rename some method, or fix license headers, or change the build, or change a version number in a build file. Don't combine multiple such things into one commit.
This helps make sure that the many different changes going on at the same time can be merged together without issues, and make it possible to go back in time to figure out what happened.
In the case of documentation files, it can make the job of the translation teams more complicated, as it becomes difficult for them to determine exactly what content changes need to be translated.
Please make your commit messages descriptive. Avoid committing changes to many files in one go with a generic, vague message. Instead, commit each file (or small, related groups of files) with tailored commit messages.
A good commit message should look like this:
Header line: Explain the commit in one line of 50 chars After a blank line, the body of commit message is a few lines of text, explaining things in more detail, possibly giving some background about the issue being fixed, etc etc. Use bullets if possible: - Line 1 - Line 2 etc. * Start works too, instead of - The body of the commit message can be several paragraphs. Use proper word-wrap and keep columns shorter than 72. That way "git log" will show things nicely even when it's indented. Include the CLOUDSTACK-xyz jira issues related to this commit, if any, somewhere in the message. Reviewed-by: Individuals who reviewed, or link to review on review.apache.org Reported-by: whoever-reported-it, if applicable (usually this is recorded in the Jira bug) Submitted-by: give credit to other individual for any patch submission committed by you Signed-off-by: Your Name <youremail@yourhost.com> |
Note that there exists a git hook to prepare a commit form for you, located at tools/git/prepare-commit-msg. One simply needs to link to it in order to use it:
ln -s ../../tools/git/prepare-commit-msg .git/hooks/prepare-commit-msg |
Please use a logical prefix, which should be:
For a feature that takes several weeks to flesh out, these guidelines may result in dozens of commits. That's absolutely fine, git's implementation means that there is basically no cost at all to keeping all that history around. Having the history available of what it took to get to a finished feature is very useful; the branch/merge history for that future provides a summary of all those different commits.
For experimental work where you have to do a lot of undo/redo work, perhaps abandoning part of a solution completely and replacing it with another, you may choose to hide some of the 'mess' of that history, creating a cleaner alternate history, pretending you got everything right the first time. Cloudstack currently doesn't have a strong rule for or against that. If you want to do it, two key commands are git rebase -i and git checkout -p. When you do it, remember: do not rebase on branches you have pushed upstream.
Be mindful of the branch that you are working in. Please use feature branches for bigger changes, and support/hotfix branches for bugfixes that have to go into multiple releases. Please be mindful that just because you have commit privileges does not relieve you of the obligation to develop in the open, which means communication on the cloudstack-dev list.
Using the git-flow command line tools is recommended but completely optional. Here's the git-flow cheat sheet: http://danielkummer.github.io/git-flow-cheatsheet/
As a committer one of your responsibilities is to review and commit patches from others. The patches must pass through reviewboard, the bug tracker or the mailing list so they can be tracked and so they can be considered a CONTRIBUTION per the apache license/CLA.
Patches should be applied via git-am (in other words, don't just use a diff, it records no author information).
See the Code Submission and Review Guidelines.
Discussion thread | Vote thread
After the 4.4.0 release, cloudstack has decided to move to (a variant of) the git-flow branching model. The switch will be made on 7th august, 2014 TBD. The setup is as follows:
This is the proposal for first cut
develop from masterrelease/4.5 from the developIdeally the continuous integration setup will evolve so that it builds and tests all branches, including all feature and bugfix branches. However that may require more resources than are currently available.
So initially, mirroring current practice, CI should build the develop branch and all active release branches.
A hotfix is a bugfix that is / has to be done to an existing release and/or release branch.
Since 4.4 can't be merged to/from master, bugfixes to 4.4 need to be done twice, once for 4.4 and once for 4.5 and beyond:
hotfix/4.4-<jira-ticket> from 4.4hotfix/<jira-ticket> from release/4.5hotfix/4.4-<jira-ticket> into 4.4hotfix/<jira-ticket> into release/4.5hotfix/<jira-ticket> into release/4.5 and pushrelease/4.5 into developFor bugfixes that don't need to go into 4.5.x, normal git-flow can be followed.
In git-flow it is customary to delete release branches after a release. We won't do that. Instead minor releases get a long-term support (LTS) release branch. This means it will be a bit easier to create patch releases for multiple releases.
Imagine we have released 4.5.0 and 4.6.0, and we want to do a hotfix that is to be part of 4.5.1 and 4.6.1:
hotfix/<jira-ticket> from release/4.5 git flow hotfix start <jira-ticket> release/4.5 mvn -P ... install && ... smoke test ... git commit -a git push upstream hotfix/<jira-ticket>hotfix/<jira-ticket> into release/4.5hotfix/<jira-ticket> into release/4.5 and push git checkout release/4.5 git pull --rebase git merge --no-ff --edit hotfix/<jira-ticket> git push upstream release/4.5develop
git checkout develop
git pull --rebase
git merge --no-ff --edit release/4.6
git push upstream develop
while it should be possible to merge your hotfix branch into develop, the possibility exists that other hotfixes were already merged into the release but not yet into develop. If you merge your hotfix but not those previous hotfixes, comparing the history of release and develop branches gets a bit more difficult. So, we merge all the hotfixes on 4.6 to develop.For bugfixes that don't need to go into 4.5.x, normal git-flow can be followed (see below).
Calling git merge with --no-ff ensures a merge commit. This commit documents the history that there was a specific hotfix.
Calling git merge with --edit allows editing the merge commit message. Use this to add any comments about the application of this hotfix to this release.
All fixes to micro releases (x.y.1, x.y.2, x.y.3, etc) must be brought in and merged as hotfixes. The only commits on micro releases that are made directly on the branch itself are those that involve making the release itself, i.e. changing the version number or the release notes.
If a bug got fixed on the develop branch, and then later on you decide that this fix has to make it into an already existing release branch, things are a little different. You do not want to merge from the develop branch into the release branch.
DON'T EVER DO THIS
git checkout release/4.5
git merge develop
This use case is what cherry-picking is for 
. Use git log (or look at github, or ...) to find the SHA1 checksum of the bugfix commit(s) that need to go onto the release branch. Then, cherry-pick it
git checkout release/4.5 git pull --rebase git cherry-pick --edit abcdef12345 git cherry-pick --edit bcdabc54321 git push upstream release/4.5
Now, release/4.5 does need to be merge-able back into develop. This will often work out ok, but if there have been many changes to develop since the cherry-picked commits, it can be harder. It's a good idea to resolve this immediately.
git checkout develop
git pull --rebase
git merge --no-ff --edit release/4.5
git push upstream develop
If that merge went without issues, you're done. Make sure to edit the merge commit message to explain what you did, since otherwise history will look just a tad weird (it will have two commits with the same commit message but different commit ids). If that merge has issues, it will need some thinking to resolve. If the conflict is just with these commits, you can probably just fix it by hand and finish the merge. If there's other problems...first abort the merge
git merge --abort
then merge release/4.5 into develop up to but excluding the changes you just made
git merge --no-ff --edit defabc543412
then merge just your recent changes into develop, using merge strategy 'ours'
git merge --no-ff --edit -s ours release/4.5
this says to resolve any conflicts by just choosing 'our' version of everything, where 'our' is the development branch. So, it is very important to apply this strategy to only the cherry-picked commits, because we do not want to miss merging any of the other hotfixes to the development tree.
...if that doesn't work, it is time to consult git merge --help, stack overflow, or the mailing list. Because you make all these changes locally, first, it is quite safe to try and experiment and see if you get the results you want. You can always roll back your changes locally. But, if things aren't working, it's best to ask for help (from a release manager, for example) rather than brute-force the change in.
See above under 'Quick Start'. This should be the default way of working for most changes.
New features should come with tests and those should go onto the feature branch.
When improving test coverage for existing functionality, you should still create a feature branch if there's a lot to the test (moving code around, refactoring to reuse existing code, changes to test tools, etc). For simply adding a new test, see below.
In the past, cloudstack has released new micro releases (x.y.1, x.y.2) that contain new features or that enable previously enabled features.
In the git-flow model, that ideally shouldn't happen. Rather the ambition should be to make the release cycle short and predictable enough that there is no pressure to push features into micro releases. Version numbers are cheap; feasible release management is more important than the psychology of the version number.
However git-flow provides a workflow model and getting to that way of working is a transition that will take time, and investment into continuous integration.
During the transition, if a decision is made to add new features to an existing release branch, the technical process for doing so is either
Following this route should be considered an exception to the rule, and should be discussed in the community. TBC a policy for when it is ok to do this.
Fixing a simple bug that
Can be done directly on develop.
However, since when you start the bugfix you typically don't know exactly what the fix will be, it's simply good practice to create a local bugfix branch anyway. Basically, whenever you do a context-switch to pick up a new task or JIRA issue, you probably want to branch. The workflow difference in this case is that the branch you create is local to your machine; you never push it upstream. For example:
Start work:
git checkout develop git pull --rebase git checkout -b bugfix/<jira-ticket> ...write test... git commit -a ...code code code... git commit -a ...test fix test... git commit -a
Check that you're done: mvn -P ... install && ... smoke test ...
Cleanup and make neat clean commit(s)
git pull
git rebase -i develop
git commit --amend
git checkout develop git merge --ff-only bugfix/<jira-ticket> git branch -d bugfix/<jira-ticket>
Push the changes:
git push upstream develop
Besides the normal git-flow commands, remember to update version numbers (in maven pom.xml, in marvin setup.py, in systemvm scripts, ...should have a confluence page...).
git checkout develop git pull --rebase git flow release start x.y develop ...update version number to x.y.0-SNASPHOT... git push upstream release/x.y git checkout develop ...update version number to x.y'-SNAPSHOT... git commit -a git push upstream develop
Don't use git flow release finish, it will delete the release branch.
git checkout release/x.y
git pull --rebase
...update version number to x.y.z...
git push upstream release/x.y
git checkout master git pull --rebase git merge --no-ff --edit release/x.y git tag -a vx.y.0 git push upstream master
git push --tags
git checkout release/x.y
...update version number to x.y.z'-SNAPSHOT...
git push upstream release/x.y
When a release branch nears release status, the release manager can declare the branch frozen. Once the branch is frozen, no-one but the release manager(s) should do any merges (or any other commits) to the branch. Committers can still create and nominate bugfix branches to merge into the release branch, but from this point on they should be extra careful to only work on those bugs for which it has been agreed they will still get fixed in that release. That agreement is reached on the development mailing list. For example, the agreement may be that fixes for JIRA blockers with a fix version of that release will be accepted, and nothing else.
This is very similar to starting minor release, but you start from the release branch, not from the develop branch.
git checkout release/x.y git pull --rebase git flow release start x.y.z release/x.y git push upstream release/x.y.z git checkout release/x.y ...update version number to x.y.z'-SNAPSHOT... git push upstream release/x.y
This is very similar to finishing a minor release. You shouldn't have to update version numbers in release/x.y in this case (since you did that at patch release start; see above), but it's probably good to check.
git checkout release/x.y.z
git pull --rebase
...update version number to x.y.z...
git push upstream release/x.y.z
git checkout master git pull --rebase git merge --no-ff --edit release/x.y.z git tag -a vx.y.z git push upstream master
git push --tags
git checkout release/x.y
...version number is already at x.y.z'-SNAPSHOT...
While changes are only in your local repository, you can do many different things to undo, amend, or rewrite commits (see git reset --help). However, once commits / tags / branches are pushed to upstream, there is no more real 'undo' and the only direction for change is forward.
To undo (a) commit(s) then, is to apply the inverse of that/those commit(s). For simple cases, see git revert --help. For complex cases, amazing amounts of tips are found on stackoverflow and elsewhere. Here's one reference.
When you have to do any work like this, the most important is that you write good commit messages detailing what you intend to do. That way, if something goes wrong while cleaning up, others can help unpick the problem.
Patches encapsulated all the changes you want to make to another branch. Patches are used to communicate the changes that you want incorporated into Apache CloudStack.
Patches should be created with git-format-patch - and you should specifically be basing that against the branch you are targeting for inclusion, so for instance using our above example from the <feature-branch> branch we'd run:
$ git format-patch -s develop
The above would produces a patch file for each commit in the directory (unless you specified one with -o /path/to/patch/folder).
If you've a series of commits that you want to put into a single file, try
$ git format-patch -s develop --stdout > ./mycommits.patch
CloudStack currently doesn't utilize Github pull requests. We ask that you post patches on Review Board.
easy_install RBToolstouch .reviewboardrcchmod go-rwx .reviewboardrccat >>.reviewboardrc <<ENDREPOSITORY = "cloudstack-git"
REVIEWBOARD_URL = "https://reviews.apache.org/"
USERNAME = "your-review-board-username"
PASSWORD = "your-review-board-password"
OPEN_BROWSER = True
TARGET_GROUPS = "cloudstack"
REPOSITORY_TYPE = "git"
# TRACKING_BRANCH = upstream/develop
END rbt post --tracking-branch=upstream/developRBTools automates creating a diff file that captures the changes you've made and publishing them to the ReviewBoard site. The drawback is that you loose the meta data associated with each commit that you've made. Howevever, Apach Review Board can't handle a commit that references the same file multiple times.
We can and occasionally do accept patches send via e-mail to the development list, or patches attached to JIRA issues, but Review Board is the standard. Please use it.
For big changes that involve multiple commits, review board unfortunately does not support patch sets. When sensible, it's preferable to create multiple reasonably-sized, reviewable commits, and multiple reviews on review board. If you've made really small/clean commits, that may be too much hassle for you and/or the reviewer. Don't spam reviewboard! In those cases, please do submit the squashed patch using post-review anyway, but then edit the description to include a link (i.e. to a branch in a cloudstack fork on github, or to a git am mbox somewhere) for communicating the full history.
If the patch to be applied gets the commit details in the header, then use
If it's just a raw patch, then use
Git assumes that if you have commit privileges that you are to be trusted, and generally this is a good thing. However, occasionally people are lured by the dark side of the force and when something doesn't merge cleanly, or their push doesn't work - they are tempted to whip out the --force and make it work. PLEASE NEVER DO THIS. It's is almost universally wrong. Please tell us about your problem on cloudstack-dev and let us help you fix it.
The coding standard for CloudStack says that unix line endings (LF) are used instead of CRLF (Windows line endings). You should set your editor and git configuration to behave properly.
For more information:
http://help.github.com/line-endings/
http://git-scm.com/docs/git-config
In short, please don't. If non-committers have a patch 95% of the way - comment and tell them what is necessary to make the patch acceptable. Let them fix their own patch and resubmit. Yes you can probably fix things more quickly, but it's important for community growth (both in number and experience) that folks do this themselves.
To install the git hooks, download and put them in the $CLOUDSTACK_SOURCE_ROOT/.git/hooks
chmod u+x may be required.
more information about git hooks is available @ http://git-scm.com/book/en/Customizing-Git-Git-Hooks