Skip to end of metadata
Go to start of metadata

Using Git with Atlas

Using Git with Atlas

Committer guidelines

These are the general guidelines that the committers should try to follow always, but can use their discretion, and rely on good faith and trust to do the right thing:

  1. All commits need a JIRA

  2. All jiras need a +1 from a committer (preferably) or subject matter expert (for things like UI - which has fewer experts in the committer list) before committing. Of course, no commit can happen if another contributor/committer has requested time for review. +1 can't be from the same person who has submitted the patch

  3. For bugs and improvements that touch core of Atlas - the type system, metadata repository, notification interface, please wait at least 18 hours from when the patch was submitted(excluding holidays), to give time for contributors/committers in other timezones to review.

  4. Features or large patches that touch many areas should have a proposal about the approach in JIRA and should be reviewed by more than one committer.

  5. Conversely, we can be a little lax with commits that touch peripheral areas that are unlikely to destabilise core - including UI, hooks, documentation and commit more freely.

  6. Exceptions are fixes for bugs that are breaking builds/tests - these can be committed without waiting, irrespective of which area they fall in. But they need +1 from some contributor, not same as patch owner.

  7. Committers should try to closely monitor activity and comments on JIRAs and proactively engage if they have issues with the approach etc.

  8. If any commit is found to break things badly, the right way to fix is to open a new JIRA stating what the issue is, caused by which JIRA and wait for the committer or contributor to respond back. A revert can be proposed depending on how bad the regression is, and the committer / contributor must be open to this in the best interest of the project.

Commit Steps

  1. Apply the patch from the JIRA on the latest codebase with no other local changes
    1. If the patch has been produced in the mail format: 
      1. git am --signoff < ATLAS_xxxx.patch
    2. If the patch has not been produced in the mail format,  you need to supply the author and message on the commit:
      1. Run git apply ATLAS_xxxx.patch
      2. Run git commit --author="<author>" --m="<ATLAS-xxxx message text>" --signoff
  2. Run 'mvn clean install'
  3. If you want to check the commits that will be pushed run 'git cherry -v'
  4. 'git push'. If git push fails because the codebase is out of sync with remote, the differences need to be resolved.
    1. If the conflicts are trivial run 'git pull --rebase'. Please don't run just 'git pull' as it adds merge commits. Resolve conflicts if any and verify the build and commit again.
    2. If the merge is not trivial, send back to the author to resolve conflicts and rerun testing and resubmit with the conflicts resolved.  
  5. Make sure the JIRA has the latest patch and resolve the JIRA with the right fixVersion
  6. Add the commit url to the Jira. To find the commit url - see https://lists.apache.org/list.html?commits@atlas.apache.org.
  7. Rectifying mistakes:
    1. Should be done in new JIRA / commits.
    2. If you commit with the wrong commit message you can alter the commit message if it is the last commit. https://help.github.com/articles/changing-a-commit-message/.

Releasing atlas

Note: Replace 0.7 with the version that you are releasing

Prepare for release

  1. Send an email to dev@ with the scope of the release and the release timelines atleast a week before the planned release start date
  2. Cut the branch 0.7-incubating
  3. Change the project versions - master branch to 0.8-incubating-SNAPSHOT and 0.7-incubating branch to 0.7-incubating
  4. Verify the source package built on 0.7-incubating - code builds with empty local mvn repo(~/.m2/repostory), tests pass, the binary package functionality

Prepare keys

  1. Generate key: http://www.apache.org/dev/openpgp.html#generate-key

  2. svn co https://dist.apache.org/repos/dist/release/incubator/atlas atlas-release

  3. Add your key to atlas/KEYS: 
    (gpg --list-sigs <your name> \
                 && gpg --armor --export <your name>) >> atlas-release/KEYS
  4. Commit atlas-release

Prepare source package

  1. Go to atlas code with the release branch
  2. Verify - build with tests succeed, the functionality works with a binary package
  3. git tag -a release-0.7-rc0 -m "Atlas 0.7-rc0 release"
  4. git push origin release-0.7-rc0
  5. mvn clean package -Pdist -DskipTests

Upload source package

  1. svn co https://dist.apache.org/repos/dist/dev/incubator/atlas atlas-dev

  2. mkdir atlas-dev/0.7-incubating-rc0
  3. cp atlas/distro/target/apache-atlas-0.7-incubating-sources.tar.gz atlas-dev/0.7-incubating-rc0/
  4. cd atlas-dev/0.7-incubating-rc0/
  5. gpg --armor --output apache-atlas-0.7-incubating-sources.tar.gz.asc --detach-sig apache-atlas-0.7-incubating-sources.tar.gz
  6. gpg --print-md MD5 apache-atlas-0.7-incubating-sources.tar.gz >apache-atlas-0.7-incubating-sources.tar.gz.md5
  7. gpg --print-md SHA512 apache-atlas-0.7-incubating-sources.tar.gz > apache-atlas-0.7-incubating-sources.tar.gz.sha512
  8. Commit atlas-dev

Voting

Vote mail template

Subject: [VOTE] Release Apache Atlas version 0.7-incubating
To: dev@atlas.incubator.apache.org
Body:
Hi Folks,

I have created a build for Apache Atlas 0.7-incubating, release candidate 0. Thanks to everyone who have contributed to this release and also tested this.

The source tarball (*.tar.gz), signature (*.asc), checksum (*.md5, *.sha):
https://dist.apache.org/repos/dist/dev/incubator/atlas/0.7.0-incubating-rc0/

The SHA512 checksum of the archive is 
51B4E097 E27A1F97 112441D2 97A0C940 282FEDF3 E864DF85 9224D533 5A00CF1E 14D86DF6
 B6E013B3 8E0A76AE 28DEE415 D6120B8D 51AD468C 46E55A28 A3613303

The commit id (ca9fea712d9a920e5949f16c32a759acd9222f19) to be voted upon:
https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=commit;h=ca9fea712d9a920e5949f16c32a759acd9222f19

The tag to be voted upon:
https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=log;h=refs/tags/release-0.7-rc0

The list of fixed issues:
https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=blob;f=release-log.txt;h=5d72620f1db75c3e032745c3f7c6cefa85b9505c;hb=refs/heads/0.7-incubating

Keys to verify the signature of the release artifact are available at:
http://www.apache.org/dist/incubator/atlas/KEYS
PGP release keys:
http://pgp.mit.edu/pks/lookup?op=vindex&search=0xA0E6F9F5D96BF0FD

Note that this is a source only release and we are voting on the source release-0.7-incubating-rc0.

Please download, test, and try it out.

Vote will be open for at least 72 hours (until the required number of IPMC votes are obtained) till 26th June, 2016 10 PM.

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove (and reason why)

+1 from my side for the release.

For folks not familiar with vetting a release, please refer to
http://incubator.apache.org/guides/releasemanagement.html#check-list

Regards,
Shwetha

If the above dev vote passes, send the result vote on dev@ and start vote on general@incubator:

To: general@incubator.apache.org
Subject: [VOTE] Release Apache Atlas version 0.7-incubating RC2
Body:
Hi all,

This is a call for a vote on the Apache Atlas 0.7-incubating release (release candidate RC2).

A vote was held on developer mailing list and it passed with 7 +1s.
Vote thread: https://s.apache.org/Czm4
Results thread: https://s.apache.org/PAeO

The source tarball (*.tar.gz), signature (*.asc), checksum (*.md5, *.sha512):
https://dist.apache.org/repos/dist/dev/incubator/atlas/0.7.0-incubating-rc2/

The SHA512 checksum of the archive is
8B923CEA 2CDD32C8 7CE9A066 38002BFA 14F666DC C56F95AF 2805D5C1 08B58F9E 436E3E3C
61B40908 474C67E7 782E720E 1BFA3678 B8099EBC 282F64C2 C16A48C4

The commit id (15748e7bc5ed019a63326f201258e3a55d512d96) to be voted upon:
https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=commit;h=15748e7bc5ed019a63326f201258e3a55d512d96

The tag to be voted upon:
https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=log;h=refs/tags/release-0.7-rc2

The list of fixed issues:
https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=blob;f=release-log.txt;hb=refs/heads/0.7-incubating

Keys to verify the signature of the release artifact are available at:
http://www.apache.org/dist/incubator/atlas/KEYS

PGP release keys:
http://pgp.mit.edu/pks/lookup?op=vindex&search=0xA0E6F9F5D96BF0FD

Note that this is a source only release and we are voting on the source release-0.7-incubating-rc2.

Please download, test, and try it out.

Vote will be open for at least 72 hours till 9th July, 2016 1 AM Pacific time.
[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove (and reason why)

Thank you!
Regards,
Hemanth

If the vote on general@ passes, send a result vote and the release is now successful.

Post release steps

  1. Publish release artifacts in apache mvn repository using instructions for branch 0.7-incubating
  2. Publish the release packages:
    1. svn co https://dist.apache.org/repos/dist/release/incubator/atlas atlas-release && cd atlas-release
    2. Delete the old release 0.6.0-incubating
    3. Copy the release artifacts from https://dist.apache.org/repos/dist/dev/incubator/atlas/0.7.0-incubating-rc2/ to new release directory 0.7.0-incubating
    4. Commit atlas-release
  3. Update atlas website documentation:
    1. Update sections Releases and Documentation in atlas code docs/src/site/site.xml - update the old release link to use archives, add sections for new release
    2. Create new directory for release in atlas-website git base - 0.7.0-incubating
    3. cp -r 0.7.0-incubating/atlas/docs/target/site/* atlas-website/0.7.0-incubating/
    4. cp -r 0.7.0-incubating/atlas/webapp/target/apidocs atlas-website/0.7.0-incubating/api
    5. cp -r master/atlas/docs/target/site/index.html atlas-website/index.html
    6. Commit atlas-website
    7. Commit atlas
  4. Send release announcement

    To: announce@apache.org; general@incubator.apache.org; dev@atlas.incubator.apache.org
    Subject: [ANNOUNCE] Apache Atlas 0.7.0-incubating released
    Body:
    Hi All,
    
    The Apache Atlas team is happy to announce the release of Apache Atlas - version 0.7.0-incubating.
    
    Atlas is a scalable and extensible set of core foundational governance
    services – enabling enterprises to effectively and efficiently meet their
    compliance requirements within Hadoop and allows integration with the whole
    enterprise data ecosystem.
    
    The release artifacts are available at:
    http://www.apache.org/dyn/closer.cgi/incubator/atlas/0.7.0-incubating/
    
    Release notes available at:
    https://git-wip-us.apache.org/repos/asf?p=incubator-atlas.git;a=blob;f=release-log.txt;hb=refs/heads/0.7-incubating
    
    To use these artifacts, please use the following documentation:
    http://atlas.incubator.apache.org/0.7.0-incubating/index.html
    
    More details on Apache Atlas can be found at:
    http://atlas.incubator.apache.org/
    
    We thank everyone who made this release possible.
    
    Thanks,
    The Apache Atlas team
    
    ----
    DISCLAIMER
    Apache Atlas is an effort undergoing incubation at the Apache
    Software Foundation (ASF), sponsored by the Apache Incubator PMC.
    
    Incubation is required of all newly accepted projects until a further
    review indicates that the infrastructure, communications, and decision
    making process have stabilized in a manner consistent with other
    successful ASF projects.
    
    While incubation status is not necessarily a reflection of the
    completeness or stability of the code, it does indicate that the
    project has yet to be fully endorsed by the ASF.


Publishing artifacts in apache mvn repository

  1. Generate PGP keys: http://www.apache.org/dev/openpgp.html#generate-key

  2. Set-up apache id credentials for maven: http://www.apache.org/dev/publishing-maven-artifacts.html#dev-env

  3. Test your settings: 'mvn clean install -Papache-release -DskipTests' (this will build artifacts, sources and sign)

  4. Deploy the artifacts: 'mvn clean deploy -Papache-release -DskipTests'
    1. For SNAPSHOT version, verify the artifacts at https://repository.apache.org/content/repositories/snapshots/org/apache/atlas/
    2. For release version, the artifacts are deployed in staging repository.
      1. Login at https://repository.apache.org/index.html, go to stagingRepositories
      2. Select the atlas repository and close. This will trigger validations on checksum and signature. If any checks fail, fix the issue, deploy the artifacts and close the repository again. If all checks succeed, release the repository
      3. Verify the artifacts at https://repository.apache.org/content/repositories/releases/org/apache/atlas/

Deploying site http://atlas.apache.org/

  1. Run 'mvn clean site' in apache/atlas
  2. git clone https://git-wip-us.apache.org/repos/asf/incubator-atlas-website.git atlas-website
  3. Go to asf-site branch
  4. cp -r atlas/docs/target/site/* atlas-website/
  5. cp -r atlas/webapp/target/apidocs atlas-website/api
  6. Verify the html files and commit
  7. Verify the site http://atlas.apache.org/

Deploying site http://incubator.apache.org/projects/atlas.html

Instructions to update an incubator site are http://incubator.apache.org/guides/website.html

 

 

 

 

 

  • No labels