This document details how to release artifacts to the staging repository. It is currently a manual process, but will be automated in the future.

There are three architectures supported OSX cpu, Linux cpu, and Linux gpu. A build and deploy must be done for each to publish the 3 artifacts.

OSX CPU 

Creating the artifact and testing it locally

First we need to modify the project file to create the correct artifact and test to make sure it is good.

• Check out the tag of the release with
  • git tag —list  - to list all the tags
  • git checkout tags/<tag_name> -b <branch_name>
• git checkout tags/<tag_name> -b <branch_name>
• Go into the clojure directory
  • cd contrib/clojure
    
• In the project.clj file, make the following edits
  • The main scala dependency jar must be edited to be the correct release and match the architecture [org.apache.mxnet/mxnet-full_2.11-osx-x86_64-cpu "1.3.0"]
  • The main project name must be changed to match the artifact that it will produce. Add -osx-cpu to the name and rename the SNAPSHOT
    • Example: (defproject org.apache.mxnet.contrib.clojure/clojure-mxnet-osx-cpu "1.3.0"
  • Make sure a license and url key are present (will be put into a PR shortly)
    

    • :url "https://github.com/apache/incubator-mxnet"
      :license {:name "Apache License"
                     :url "http://www.apache.org/licenses/LICENSE-2.0"}

• From the command line, verify that the Scala dependency is downloaded and the tests are run by running lein test

The tests should run without error

• From the command line, run lein install, which will create the jar in the target directory and install it in your local ~/.m2 directory
  • Verify the jar structure in target and make sure it looks right
  • Optionally, you can test this by
    • cd examples/imclassification
    • do lein run to use the jar and run the example

Deploying the artifact to staging

Now that we know we are producing a good jar, we can move to deploy it. There are two authentication parts to deploying, your ASF login used to login to the repository and your GPG signature that is used to sign the release. 

The leiningen build tools handles configuration for both these parts. Good resources to review are:

https://github.com/technomancy/leiningen/blob/master/doc/DEPLOY.md

https://github.com/technomancy/leiningen/blob/master/doc/GPG.md

The first thing you will need is to create a credentials file for leiningen - this will be encypted with gpg

• Create a file ~./lein/credentials.clj file with the contents :
  • {#"staging" {:username "user"
    :password "pass"}}
• Encypt that file with gpg --sign ~/.lein/credentials.clj 
  • You can test decrypting with gpg -d ~/.lein/credentials.clj.gpg
  • Delete the original unencrypted ~/.lein/credentials.clj

The next thing you will need is to modify the project.clj file to have the ASF deploy staging repo and also point to your ASF gpg key.

• Add the following to your file

  •  :deploy-repositories [["staging" {:sign-releases true
             :url "https://repository.apache.org/service/local/staging/deploy/maven2"
            :creds :gpg
            :signing {:gpg-key "your key"} ;; this can be email of key-id
           :snapshots true
           :checksum :fail
           :update :always
           :releases {:checksum :fail :update :always}}]] 

  • Also change the other respository name from staging to staging-downloads so there is no confusion

  • if it does not prompt for passphrase (known bug in OSX) - you can run 

    gpg --quiet --batch --decrypt ~/.lein/credentials.clj.gpg

• Now you can run the deploy with lein deploy staging. You will be prompted for you gpg passphrase.

  • For automatic deploys in the future, we can look into more why this is prompting. It does not prompt for linux

    
    

  • If you run into the error "gpg: signing failed: Inappropriate ioctl for device" - Run export GPG_TTY=$(tty)

Locate and Examine the Staging Repository

DO NOT PUSH RELEASE - IT WILL SEND IT TO RELEASES AND MAVEN CENTRAL!

If all went well with the deploy step, you need to login to the repository and verify the upload and then close it. See this document https://central.sonatype.org/pages/releasing-the-deployment.html#locate-and-examine-your-staging-repository for more details.

If you run into an error closing, it will show up here. For example, your apache gpg public key needs to be uploaded here http://pgp.mit.edu:11371/ or at another public server to verify the release.

Final testing of downloading the jar from staging and running example

To give the jar a final test, you can delete it from your local ~/.m2 cache and try to run the example again with

• cd examples/imclassification
• add the repository in the project.clj to pull from staging

  • :repositories [["staging-download" {:url "https://repository.apache.org/content/groups/staging"
    :snapshots true
    :checksum :fail
    :update :always
    :releases {:checksum :fail :update :always}}]]

• do lein run to use the jar and run the example - You should see the dependency download from staging
  

  • Retrieving org/apache/mxnet/contrib/clojure/clojure-mxnet-osx-cpu/1.3.0/clojure-mxnet-osx-cpu-1.3.0.pom from staging-download

    
    

Linux CPU 

To build for linux-cpu platform and gpu platforms, we use the AWS Deep Learning Base AMI that comes built with all the necessary 

libraries such as opencv/cudnn etc,.

Export/Import GPG Keys

First you need to export the gpg keys you created on Mac and import to the linux host

Install Maven, gnupg, gnupg2 & gnupg-agent

Export and ship gpg keys you created on the Mac

Import and verify they look fine

Update the gnupg-agent cachettl, it takes a long time to build on linux and linux-gpu instances and will time out during release if you don't baby sit the build

Start gpg-agent to use the above

Creating the artifact and testing it locally

Follow the same instructions as for OSX but use  clojure-mxnet-linux-cpu for the project name

Deploying the artifact to staging

Follow the same instructions as for OSX.

If you run into trouble with it prompting you or not prompting you for signing key use this:

export GPG_TTY=$(tty)

Locate and Examine the Staging Repository

same process as OSX

Final testing of downloading the jar from staging and running example

Same process as OSX with correct linux-cpu artifact

Linux GPU 

You can do this in the same instance as the linux cpu since you have all the keys setup

Creating the artifact and testing it locally

Follow the same instructions as for OSX but use  clojure-mxnet-linux-gpu for the project name

Deploying the artifact to staging

Follow the same instructions as for OSX but you won't need to recreate your credentials.clj.gpg file

If you run into trouble with it prompting you or not prompting you for signing key use this:

export GPG_TTY=$(tty)

Locate and Examine the Staging Repository

same process as OSX

Final testing of downloading the jar from staging and running example

Same process as OSX with correct linux-gpu artifact

What to do if you accidentally release the jar

• Don't panic
• Mail dev list about it and find out how they want to proceed
• To clean up you will need to coordinate with infra
  • Create an infra ticket - to ask for the cleanup to repository.apache.org
    • Reference the dev list thread
  • Create a ticket to issues.sonatype.org to delete it from central 
  • Here is an unfortunate example  INFRA-17905 - Getting issue details... STATUS
• Figure out what steps to do to prevent it in the future