Note: Following instructions are applicable for publishing Scala and Java package simultaneously as both bindings are inside the same package and built together.
Currently 3 Packages are published to Maven. `mxnet_full_2.11-{platform}` for the following platforms: linux-x86_64-cpu, linux-x86_64-gpu, osx-x86_64-cpu.
Setup GPG Keys https://www.apache.org/dev/openpgp.html#key-gen-install-latest-gnupg. Make sure to follow https://www.apache.org/dev/openpgp.html#update to send your key to Apache and onto the public PGP Keyservers. You also need to add your key to both the KEYS file on mxnet github and the KEYS file in the apache committers subversion.
git clone https://github.com/apache/incubator-mxnet ~/mxnet-$RELEASEVERSION cd ~/mxnet-$RELEASEVERSION git checkout $RESOURCE_CANDIDATE_TAG |
#try opencv3 next time you release brew install lapack openblas opencv |
Now setup your maven create a master password and paste the encrypted the more info Password Encryption
touch ~/.m2/settings-security.xml && mvn --encrypt-master-password |
Paste the encrypted master-password you get from above in `~/.m2/settings-security.xml`
<settingsSecurity> <master>ENCRYPTEDMASTERPASSWORD</master> </settingsSecurity> |
Encrypt your password for `apache` login, note that this is not the passphrase of your gpg-key.
cp /usr/share/maven/conf/settings.xml ~/.m2/ && mvn --encrypt-password |
Paste the below with your apache username and encrypted apache password updated under `<servers>`tag, this will used to publish to apache snapshot and staging in `~/.m2/settings.xml`.
<!-- To publish a snapshot of some part of Maven -->
<server>
<id>apache.snapshots.https</id>
<username>APACHEUSERNAME</username>
<password>APACHEPASSWORDENCRYPTED</password>
</server>
<!-- To stage a release of some part of Maven -->
<server>
<id>apache.releases.https</id>
<username>APACHEUSERNAME</username>
<password>APACHEPASSWORDENCRYPTED</password>
</server> |
Edit the file `ci/publish/scala/build.sh` and add the flag `-Pstaging` to the mvn deploy command. Make sure to have your GPG key available on the machine. If you have multiple GPG keys, you can add `-Dgpg.keyname=$DESIRED_KEY_EMAIL` to the mvn deploy command to specify the desired key.
#if you don't set this it will fail with the error "gpg: signing failed: Inappropriate ioctl for device" export GPG_TTY=$(tty) # Set MXNet variant to build export mxnet_variant=mkl # Run the build bash ci/publish/scala/build.sh |
Once you have finished, you should be able to find the newly added version on the Apache Nexus at https://repository.apache.org/#stagingRepositories. It will have repository name orgapachemxnet-$RELEASE_NUMBER. Click on the repository and press the close button at the top to indicate that everything has been uploaded. At this point, Nexus will verify that all of the checksums are correct and the signatures are all valid (including that you have properly uploaded your key and are authorized to sign the package). Then, follow the instructions to build the CPU/GPU packages and close them before continuing to the end.
Once you have tested the staged package, you can promote the package and it will become available on maven central, find details here
To build for linux, we use a Ubuntu 14.04 instance. Install the necessary dependencies by running the install script found in the MXNet repo `ci/docker/install/ubuntu_publish.sh`.
First you need to export the gpg keys you created on Mac and import to the linux host
Install Maven, gnupg, gnupg2 & gnupg-agent
sudo apt-get install -y maven gnupg gnupg2 gnupg-agent && #soft link gnupg2 to gnupg, Update the below when you find a better solution sudo mv /usr/bin/gpg /usr/bin/gpg1 && sudo ln -s /usr/bin/gpg2 /usr/bin/gpg |
Export and ship gpg keys you created on the Mac
gpg --export-secret-keys -a xxx@apache.org > ~/Downloads/keys.asc scp ~/Downloads/keys.asc ubuntu@host:~/keys.asc |
Import and verify they look fine
gpg --import ~/keys.asc #note the key id gpg --list-keys |
Update the gnupg-agent cachettl, it takes a long time to build on linux and linux-gpu instances and will time out during release if you don't baby sit the build
mkdir -p ~/.gnupg && touch ~/.gnupg/gpg-agent.conf #set to 4 hours echo "default-cache-ttl 14400" > ~/.gnupg/gpg-agent.conf && echo "max-cache-ttl 14400" >> ~/.gnupg/gpg-agent.conf #key id you go when you ran gpg --listkeys. This isn't working, commenting out #echo "default-key KEYIDFROMABOVE" >> ~/.gnupg/gpg-agent.conf |
Start gpg-agent to use the above
eval $(gpg-agent --daemon) |
Follow steps to clone the repo and checkout the correct release branch. We recommend using a clean copy of the repo for the CPU and GPU builds
git clone https://github.com/apache/incubator-mxnet ~/mxnet-$RELEASEVERSION cd ~/mxnet-$RELEASEVERSION git checkout $RESOURCE_CANDIDATE_TAG |
Edit the file `ci/publish/scala/build.sh` and add the flag `-Pstaging` to the mvn deploy command. Make sure to have your GPG key available on the machine. If you have multiple GPG keys, you can add `-Dgpg.keyname=$DESIRED_KEY_EMAIL` to the mvn deploy command to specify the desired key. Alter the script below to set the mxnet_variant to the desired version to build. We currently use `mkl` for the CPU build and `cu92mkl` for the GPU build.
#if you don't set this it will fail with the error "gpg: signing failed: Inappropriate ioctl for device" export GPG_TTY=$(tty) # Set MXNet variant to build export mxnet_variant=$DESIRED_VARIANT # Run the build bash ci/publish/scala/build.sh |
Once you have finished, you should be able to find the newly added version on the Apache Nexus at https://repository.apache.org/#stagingRepositories. It will have repository name orgapachemxnet-$RELEASE_NUMBER. Click on the repository and press the close button at the top to indicate that everything has been uploaded. At this point, Nexus will verify that all of the checksums are correct and the signatures are all valid (including that you have properly uploaded your key and are authorized to sign the package). Then, follow the instructions to build the CPU/GPU packages and close them before continuing to the end.
At this point, you should have three packages that are closed on the apache nexus. You should now have other members of the community verify these closed packages. You can find the packages for download in a Maven repository format (which can be included following a process like https://maven.apache.org/guides/mini/guide-multiple-repositories.html) at a URL similar to https://repository.apache.org/service/local/repositories/orgapachemxnet-$BUILD_NUM/content. The $BUILD_NUM can be found as the name of the staged repository on Nexus. Note that you would have one of these repositories for each of the three builds. These URLs should be shared to help the community members verify the packages and to help build the Clojure release jars.
Following the results of the vote, you should use Nexus to finish the release process. When you go to the staging repository and view the package, you will have a release button and a drop button. Click the release button if the vote passed to finalize the release and make the package available through Maven Central (this can not be reversed). Click the Drop button if the vote failed as the staged package will not be released. If you have to make a release candidate, it will create new staged items with incremented staging numbers instead of changing the old ones.
sudo apt-get install gnupg2 sudo mv /usr/bin/gpg /usr/bin/gpg.1 sudo ln -s /usr/bin/gpg2 /usr/bin/gpg |
If you have an error during the release and it makes it to the Maven part of the build process, you only need to run the mvn deploy command from `ci/publish/scala/build.sh` instead of re-running the whole build. However, the GPU build requires copying some environment variables that are created during the build. Those can be found in `tools/staticbuild/build.sh` and then `tools/setup_gpu_build_tools.sh`.
Make sure you are using the encrypted password of your apache login and not gpg passphrase
Verify that the released artifacts of the version you are publishing are synced to Apache release repository and Maven Central Repository, It might be that Maven Search is not picking up.
do a make clean and make scala clean before you build, it might be that the size of the Jar is too big and its splitting across two repos.
Apache Release Repo
https://repository.apache.org/content/repositories/releases/org/apache/mxnet/
Maven Central Repo
http://central.maven.org/maven2/org/apache/mxnet/mxnet-full_2.11-osx-x86_64-cpu/1.2.1/
http://central.maven.org/maven2/org/apache/mxnet/mxnet-full_2.11-linux-x86_64-cpu/1.2.1
http://central.maven.org/maven2/org/apache/mxnet/mxnet-full_2.11-linux-x86_64-gpu/1.2.1/