Page History

Getting Started

We use conda-forge to host the ocw package. To get started, you should fork our feedstock repo. The only file you will need to concern yourself with is recipe/meta.yaml. 

The meta.yaml recipe file

There are only a few entries in this file that should be updated. These are as follows:

Package Version

This should be in the form X.Y.Z which corresponds to the latest release. Note that the ocw source distribution for the given version must first be uploaded to PyPI before making this change, or else conda will fail to build the package. This is main field that you need to change when creating a new release.

Ex

conda config --add channels agoodman

Now you can perform any of the above commands without needing to specify '-c agoodman'. 

To upload your packages to the main OCW channel, you will need to obtain a secure token which you may request by sending an email to the dev mailing list (dev@climate.apache.org). If approved, you may update your config with:

conda{% configset --addversion channels https://conda.anaconda.org/t/<TOKEN>/agoodman

Where <TOKEN> is the token string.

Updating and Building the OCW conda package

Edit the recipe configuration file

= "X.Y.Z" %}

Where X.Y.Z should be replaced by the version number of the release.

Hash

We use sha256 to populate the hash field. This too must be changed with every new release. To do this, you'll need to make sure OpenSSL is installed on your machineThe main recipes for the OCW package (and the non-default conda dependencies) can be found in the OCW codebase. Navigate to the conda_recipes directory:

cd climate/conda_recipes

conda install openssl

Then to generate the checksum, you may useEach subdirectory contains the recipes specific to each package, and these too will be necessary to edit, rebuild, and upload to the OCW channel if we wish to keep everything up to date. For now though we will focus our example workflow on how to update the main ocw package itself. Our hypothetical case will involve creating a package for version 1.1.0. Open ocw/meta.yaml and edit the following entries:

#pip Underdownload package:
version: 1.1.0
 
# Under source:
git_rev: 1.1.0

Here of course the version numbers could be set to whatever release you want. Leaving the git_rev entry commented out will fetch the latest code from the git repo rather than a stable release. Avoid doing this unless you are sure the latest revision of the code is about to be released. If needed, you should also add any new dependencies under the run section.

Build and test the package

ocw
openssl sha256 ocw-X.Y.Z.tar.gz

Where X.Y.Z should be replaced by the version number of the release. Then you may use the resulting checksum to set the hash in the recipe file. For exampleYou should now save your changes and prepare to build the package. If you do not already have it installed, you will first need to obtain the latest version of conda build:

conda update conda
conda install conda-build

{% set sha256 = "8f12405ec02ea327b1e0cf65547e409232f67577be6eb348d2ea9011827e2c4a" %}

Build Section

A few notes:

• number should be set to 0 for all new releases. For updating an existing release, increment this by 1 for each update.

• skip Should only be set to True for unsupported platforms. This can be achieved using preprocessing selectors. For example, to skip building for Python 3, you would use:

  Code Block
  language bash

conda build ocw

If successful, your newly built package will be found in a tarball located in your conda environment's <conda_environment>/conda-bld/<platform> directory. For an anaconda user's default environment on Mac OSX, this would be located in ~/anaconda/conda-bld/osx-64. 

• skip: True # [py3k]

  Changes in compatibility between platforms or python versions may require you to update the CI build configuration files as well. See re-rendering the feedstock for more information.

• Do not change the script field!

Requirements Section

This section lists all of the dependencies needed for ocw at build time (not really important since we don't use any extensions) and runtime (much more important). As with the skip field in the build section, certain dependencies can and should be excluded if they are not supported on certain platforms. For example, if a package named "foo" is not compatible with Windows, it may be excluded usingBefore proceeding, you should (preferably in a clean environment) install and test your newly built package locally:

conda   clean -t -p
conda install --use-local ocw

If everything checks out, you are finally ready to move on to the next step.

Uploading packages to the OCW conda channel

Convert the package to other platforms

Having successfully built a package on your system, you should first add versions for all available platforms. The Mac OSX anaconda user in our example would do this with:

conda convert --platform all ~/anaconda/conda-bld/osx-64/ocw-1.1.0-py27_0.tar.bz2 -o ~/anaconda/conda-bld

The directory structure here should be altered for the name of your base conda directory, OS platform and package tarball, of course.

Uploading your package

To begin uploading your packages, create an account for the anaconda cloud if you have not already done so. You will also need to install the anaconda client:

conda install anaconda-client

Then log in with your account info:

anaconda login

Finally to upload our OSX package as per our example, use:

anaconda upload -u agoodman ~/anaconda/conda-bld/osx-64/ocw-1.1.0-py27_0.tar.bz2

Where again the location should point to your desired package tarball and platform. You should repeat this process for each platform.

Sharing your updated recipe files

foo # [not win]

Maintainers Section

This section contains a list of GitHub usernames which have commit rights to the ocw-feedstock repo. If you are a regular contributor to the project and want to help maintain the recipe file, you should add your own GitHub username to this list.

Guidelines for updating the recipe

• Any updates to the recipe should be done through a pull request to the feedstock repo. 
• All changes should be done from your own fork (not on an upstream branch). 
• Finally, make sure that all of the CI builds (Circle, AppVeyor, and Travis) are passing before merging the PR. While this might seem inconvenient for committing very trivial changes to the recipe, it is necessary because it is the very same CI services which automate the process of building and uploading the binary package files to the conda-forge channel. 

Re-rendering the Feedstock

Introduction

So far in our discussion of packaging we have only needed to be concerned with the meta.yaml recipe file. However it is also important to understand that conda-forge, in its efforts to support as many platforms as possible, uses CI services (remote virtual machines) to actually build the packages from the recipe. These are Circle (GNU/Linux), AppVeyor (Windows) and Travis (OSX), which in the feedstock repo are configured respectively in circle.yml, appveyor.yml, and .travis.yml.

Generally speaking, the conda-forge webservices should automatically update these files as needed. This is typically done with periodic "maintenance" pull requests in the feedstock repo made by the conda-forge-admin account (which maintainers of this project are then responsible for merging). We refer to this process as re-rendering the feedstock, as the actual CI configuration files are generated from the jinja2 templates found here. 

Manually re-rendering the feedstock

While the process of re-rendering should happen automatically most of the time (as described above), there are a few situations where it will be necessary for you to do it manually. These can include but are not limited to the following:

• Changing supported platforms (eg, GNU/Linux, OSX, and Windows)
• Changing supported python versions (2.7, 3.5, and/or 3.6)
• Your CI builds have errors which are not related to the recipe file itself (very rare, here is an example)

In order to rerender the feedstock, you will need to use conda-smithy. Please follow the instructions in the link and make sure your version of conda-build is up to date as well. We also recommend using the --commit option with conda-smithy as this will automatically commit the re-rendered files to the feedstock repo with the conda-smithy version in the commit message. Afterwards you may push the changes to your fork and make a pull request (or to your release branch if you are doing the re-rendering on top of a new release)Changes to the recipe files should be committed to our git repo with each release, especially if dependencies are changed.