Using Git with Atlas

Using Git with Atlas

Committer guidelines

These are the general guidelines that the committers should try to follow always, but can use their discretion, and rely on good faith and trust to do the right thing:

  1. All commits need a JIRA

  2. All jiras need a +1 from a committer (preferably) or subject matter expert (for things like UI - which has fewer experts in the committer list) before committing. Of course, no commit can happen if another contributor/committer has requested time for review. +1 can't be from the same person who has submitted the patch

  3. For bugs and improvements that touch core of Atlas - the type system, metadata repository, notification interface, please wait at least 18 hours from when the patch was submitted(excluding holidays), to give time for contributors/committers in other timezones to review.

  4. Features or large patches that touch many areas should have a proposal about the approach in JIRA and should be reviewed by more than one committer.

  5. Conversely, we can be a little lax with commits that touch peripheral areas that are unlikely to destabilise core - including UI, hooks, documentation and commit more freely.

  6. Exceptions are fixes for bugs that are breaking builds/tests - these can be committed without waiting, irrespective of which area they fall in. But they need +1 from some contributor, not same as patch owner.

  7. Committers should try to closely monitor activity and comments on JIRAs and proactively engage if they have issues with the approach etc.

  8. If any commit is found to break things badly, the right way to fix is to open a new JIRA stating what the issue is, caused by which JIRA and wait for the committer or contributor to respond back. A revert can be proposed depending on how bad the regression is, and the committer / contributor must be open to this in the best interest of the project.

Commit Steps

  1. Apply the patch from the JIRA on the latest codebase with no other local changes
    1. If the patch has been produced in the mail format: 
      1. git am --signoff < ATLAS_xxxx.patch
    2. If the patch has not been produced in the mail format,  you need to supply the author and message on the commit:
      1. Run git apply ATLAS_xxxx.patch
      2. Run git commit --author="<author>" --m="<ATLAS-xxxx message text>" --signoff
  2. Run 'mvn clean install'
  3. If you want to check the commits that will be pushed run 'git cherry -v'
  4. 'git push'. If git push fails because the codebase is out of sync with remote, the differences need to be resolved.
    1. If the conflicts are trivial run 'git pull --rebase'. Please don't run just 'git pull' as it adds merge commits. Resolve conflicts if any and verify the build and commit again.
    2. If the merge is not trivial, send back to the author to resolve conflicts and rerun testing and resubmit with the conflicts resolved.  
  5. Make sure the JIRA has the latest patch and resolve the JIRA with the right fixVersion
  6. Add the commit url to the Jira, run git log and copy and paste the commit information into the Jira. You can add this text at the same time as you resolve the Jira (step 6). 
  7. Rectifying mistakes:
    1. Should be done in new JIRA / commits.
    2. If you commit with the wrong commit message you can alter the commit message if it is the last commit. https://help.github.com/articles/changing-a-commit-message/.

Releasing atlas

Note: Replace 0.8.1 with the version that you are releasing

Prepare for release

  1. Send an email to dev@ with the scope of the release and the release timelines atleast a week before the planned release start date
  2. Create a new branch for the release, if necessary.
  3. Update Atlas version in all pom files (to the release version)
  4. Verify the sources in the release branch with the following steps:
    1. builds cleanly with empty local mvn repo(~/.m2/repostory): mvn clean install && 
    2. packaging & distro creation completes successfully: mvn clean package -Pdist,embedded-hbase-solr
    3. Atlas installed with the created binary package is functional

Prepare keys

  1. Generate new PGP key: http://www.apache.org/dev/openpgp.html#generate-key

  2. Checkout atlas release repo using svn:

    svn co https://dist.apache.org/repos/dist/release/atlas atlas-release
  3. Add your key to atlas/KEYS: 
    (gpg --list-sigs <your name> \
             && gpg --armor --export <your name>) >> atlas-release/KEYS
    e.g.
    (gpg --list-sigs "John Smith" \ 
                 && gpg --armor --export "John Smith") >> atlas-release/KEYS

  4. Verify KEYS file to see if your PGP key has been added successfully, svn commit modified KEYS file in atlas-release:
     cd atlas-release
     svn add KEYS
     svn commit KEYS -m "Added GPG Key for <your name> to KEYS file" --username <apache user-id>

Prepare source package

  1. Go to atlas code with the release branch

  2. git tag -a release-0.8.1-rc0 -m "Atlas 0.8.1-rc0 release"

  3. git push origin release-0.8.1-rc0
  4. mvn clean package -Pdist -DskipTests

Upload source package

  1. svn co https://dist.apache.org/repos/dist/dev/atlas atlas-dev

  2. mkdir atlas-dev/0.8.1-rc0
  3. cp atlas/distro/target/apache-atlas-0.8.1-sources.tar.gz atlas-dev/0.8.1-rc0/
  4. cd atlas-dev/0.8.1-rc0/
  5. gpg --armor --output apache-atlas-0.8.1-sources.tar.gz.asc --detach-sig apache-atlas-0.8.1-sources.tar.gz
  6. gpg --print-md MD5 apache-atlas-0.8.1-sources.tar.gz >apache-atlas-0.8.1-sources.tar.gz.md5
  7. gpg --print-md SHA512 apache-atlas-0.8.1-sources.tar.gz > apache-atlas-0.8.1-sources.tar.gz.sha512
  8. Commit atlas-dev

Voting

Vote mail template

Subject: [VOTE] Release Apache Atlas version 0.8.1
To: dev@atlas.apache.org
Body:
Atlas team,

Apache Atlas 0.8.1 release candidate #0 is now available for a vote within dev community. Links to the release artifacts are given below. Please review and vote.

The vote will be open for at least 72 hours or until necessary votes are reached.
  [ ] +1 approve
  [ ] +0 no opinion
  [ ] -1 disapprove (and reason why)
 
 
Thanks,
Madhan


List of issues addressed in this release: https://issues.apache.org/jira/issues/?jql=project%20%3D%20Atlas%20AND%20fixVersion%20%3D%200.8.1%20ORDER%20BY%20key%20DESC
 
Git tag for the release: https://github.com/apache/atlas/tree/release-0.8.1-rc0
 
Sources for the release: https://dist.apache.org/repos/dist/dev/atlas/0.8.1-rc0/apache-atlas-0.8.1-sources.tar.gz
 
Source release verification:
  PGP Signature: https://dist.apache.org/repos/dist/dev/atlas/0.8.1-rc0/apache-atlas-0.8.1-sources.tar.gz.asc
  MD5 Hash: https://dist.apache.org/repos/dist/dev/atlas/0.8.1-rc0/apache-atlas-0.8.1-sources.tar.gz.md5
 
Keys to verify the signature of the release artifacts are available at: https://dist.apache.org/repos/dist/dev/atlas/KEYS
 

If the above vote passes, send a result vote and the release is now successful.

Post release steps

  1. Publish release artifacts in apache mvn repository using instructions
  2. Publish the release packages:
    1. svn co https://dist.apache.org/repos/dist/release/atlas atlas-release && cd atlas-release
    2. Delete the old releases
    3. Copy the release artifacts from https://dist.apache.org/repos/dist/dev/atlas/0.8.1-rc0/ to new release directory 0.8.1
    4. Commit atlas-release
  3. Update atlas website documentation:

    1. Atlas source document changes:

      1. Switch to release branch: git checkout branch-2.0

      2. Generate Java/REST API docs from Atlas sources with the following command from atlas root directory: mvn clean -DskipTests -DskipEnunciate=false package

      3. Make necessary .md changes in "atlas/docs" module

        1. Add a new page inside Whats-New folder to list new features in the current release - "What's new in Apache Atlas 2.1?"

        2. Update Downloads.md to add the release artifacts (source, signature and SHA512) and add release notes for current release.

        3. Add the version entry in config/versions.js to list new version in the dropdown.

        4. Update Overview.md to update links to "What's new in Apache Atlas 2.1?" page.

      4. Generate documentation from .md sources with the following command from "atlas/docs" directory: mvn clean site

      5. To validate the generated document execute the command:  "mvn site:run"

      6. Validate generated Atlas Doc and API docs; if looks good commit changes to branch-2.0 (release branch) and master branch.

    2. Atlas Website changes:

      1. Clone Atlas Website sources: git clone https://gitbox.apache.org/repos/asf/atlas-website.git atlas-website
      2. Switch to asf-site branch: cd atlas-website; git checkout asf-site; cd ../;
      3. Create following directories for release contents: mkdir -p atlas-website/2.0.0/api/v2
      4. Copy documentation from atlas source generated earlier to Atlas website repository: cp -r atlas/docs/target/site/* atlas-website/2.0.0/
      5. Copy Java/REST API doc from atlas source generated earlier to Atlas website repository: cp -r atlas/webapp/target/api/v2/apidocs/* atlas-website/2.0.0/api/v2/
      6. Update site's root documentation with the latest version contents: cp -r atlas-website/2.0.0/ atlas-website/
      7. Verify html files and commit
      8. Verify site http://atlas.apache.org

  4. Send release announcement

    To: announce@apache.org, dev@atlas.apache.org, user@atlas.apache.org

Subject: [ANNOUNCE] Apache Atlas 0.8.1 released
Body:
Hi All,

The Apache Atlas team is happy to announce the release of Apache Atlas - version 0.8.1.

Atlas is a scalable and extensible set of core foundational governance
services – enabling enterprises to effectively and efficiently meet their
compliance requirements within Hadoop and allows integration with the whole
enterprise data ecosystem.

The release artifacts are available at: https://atlas.apache.org/#/Downloads

The binary artifacts are available from Maven central and its mirrors.

To use these artifacts, please use the following documentation: http://atlas.apache.org/0.8.1/index.html

More details on Apache Atlas can be found at: http://atlas.apache.org/

We thank everyone who made this release possible.

Thanks,
Apache Atlas team


Publishing artifacts in apache mvn repository

  1. Generate PGP keys: http://www.apache.org/dev/openpgp.html#generate-key

  2. Set-up apache id credentials for maven: http://www.apache.org/dev/publishing-maven-artifacts.html#dev-env

  3. Test your settings: 'mvn clean install -Papache-release -DskipTests -DskipDocs' (this will build artifacts, sources and sign)

  4. Deploy the artifacts: 'mvn clean deploy -Papache-release -DskipTests -DskipDocs'
    1. For SNAPSHOT version, verify the artifacts at https://repository.apache.org/content/repositories/snapshots/org/apache/atlas/
    2. For release version, the artifacts are deployed in staging repository.
      1. Login at https://repository.apache.org/index.html, go to stagingRepositories
      2. Select the atlas repository and close. This will trigger validations on checksum and signature. If any checks fail, fix the issue, deploy the artifacts and close the repository again. If all checks succeed, release the repository
      3. Verify the artifacts at https://repository.apache.org/content/repositories/releases/org/apache/atlas/




  • No labels