This Confluence has been LDAP enabled, if you are an ASF Committer, please use your LDAP Credentials to login. Any problems file an INFRA jira ticket please.

Child pages
  • Installing OCW using the conda package manager
Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 5 Next »

Introduction

conda is an easy to use and maintain software package management system. It provides a simple and painless method for installing OCW and its dependencies across a variety of platforms.

OCW Installation Instructions

Requirements

  • The latest version of conda. The recommended way to obtain this is to install either the Anaconda or Miniconda scientific python distributions. Be sure to allow the installation to update your PATH for you. 

Installation Information

In addition to the standard scientific python packages bundled with most anaconda installations, the following packages will be installed:

  • requests: Used to make HTTP requests. The OCW UI and Toolkit use this.
  • bottle: Simple Python backend webserver used by the OCW UI.
  • pydap: A Python library for connecting to OpenDAP servers. The Toolkit uses this to handle OpenDAP connections.
  • webtest: A Python library to improve testing of web applications. The OCW UI backend uses this for testing.
  • pyesgf: A Python library for downloading files from the Earth System Grid Federation database.

How To

Before invoking conda to install the dependencies, it's always a good idea to ensure that your version of conda is up to date. You can do this with:

conda update conda

Then to install OCW and its dependencies into your conda environment (by default in ~/anaconda or ~/miniconda), you may use:

conda install -c agoodman ocw

Testing

To give yourself some peace of mind that such a simple installation method actually works, it is recommended that you test you installation. The main OCW codebase has some simple examples which may be obtained using:

cd
git clone https://github.com/apache/climate.git

If you do not wish to directly use git, you may download and extract a copy of the code directly from our GitHub page. Once you've obtained the code, head on over to the examples and run one of them.

cd climate/examples
python simple_model_to_model_bias.py

After the evaluation runs you should find a .png file in the examples directory. Congratulations, your install was successful!

Fetching Updates and changing versions

Keeping OCW up to date is even more simple than the installation itself. Simply use:

conda update -c agoodman ocw 
What if you wish to use a particular release of OCW, say version 1.1.0? Simply specify the version in the install command as follows:

 

conda update -c agoodman ocw=1.1.0 

 

Creating a clean environment

It should be noted that the above instructions will install OCW and its dependencies into your default conda environment. For those users who wish to perform development and testing in a clean and isolated environment, they should instead perform their installations using:

conda create -c agoodman -n ocw ocw

This will create a new virtual environment named "ocw" with only OCW and its dependencies installed. To activate this environment, use:

Linux/OSX
source activate ocw
Windows
activate ocw

To deactivate the environment, use:

Linux/OSX
source deactivate
Windows
deactivate

See this page for further documentation on virtual conda environments.

Maintaining the conda packages (for advanced users)

The following section contains some useful information for those in the OCW developer community who are interested in maintaining the conda packages.

Adding the OCW channel to your conda configuration

You may have noticed that many of the conda commands entered above include a -c flag. This is because the OCW package as well as some of its dependencies cannot be found in the default conda channels,  making it necessary for us to host them on a separate channel. Since adding this information to each command can get cumbersome, we may simply add it to our configuration as follows:

 

conda config --add channels agoodman

Now you can perform any of the above commands without needing to specify '-c agoodman'. 

To upload your packages to the main OCW channel, you will need to obtain a secure token which you may request by sending an email to the dev mailing list (dev@climate.apache.org). If approved, you may update your config with:

conda config --add channels https://conda.anaconda.org/t/<TOKEN>/agoodman

Where <TOKEN> is the token string.

Updating and Building the OCW conda package

It is highly recommended that you read the conda build documentation first.

Edit the recipe configuration file

The main recipes for the OCW package (and the non-default conda dependencies) can be found in the OCW codebase. Navigate to the conda_recipes directory:

cd climate/conda_recipes

Each subdirectory contains the recipes specific to each package, and these too will be necessary to edit, rebuild, and upload to the OCW channel if we wish to keep everything up to date. For now though we will focus our example workflow on how to update the main ocw package itself. Our hypothetical case will involve creating a package for version 1.1.0. Open ocw/meta.yaml and edit the following entries:

# Under package:
version: 1.1.0
 
# Under source:
git_rev: 1.1.0

Here of course the version numbers could be set to whatever release you want. Leaving the git_rev entry commented out will fetch the latest code from the git repo rather than a stable release. Avoid doing this unless you are sure the latest revision of the code is about to be released. If needed, you should also add any new dependencies under the run section.

Build and test the package

You should now save your changes and prepare to build the package. If you do not already have it installed, you will first need to obtain the latest version of conda build:

conda update conda
conda install conda-build

To build the package, being sure you are in the conda_recipes directory, use:

conda build ocw

If successful, your newly built package will be found in a tarball located in your conda environment's <conda_environment>/conda-bld/<platform> directory. For an anaconda user's default environment on Mac OSX, this would be located in ~/anaconda/conda-bld/osx-64. 

Before proceeding, you should (preferably in a clean environment) install and test your newly built package locally:

conda install --use-local ocw

If everything checks out, you are finally ready to move on to the next step.

Uploading packages to the OCW conda channel

Convert the package to other platforms

Having successfully built a package on your system, you should first add versions for all available platforms. The Mac OSX anaconda user in our example would do this with:

conda convert --platform all ~/anaconda/conda-bld/osx-64/ocw-1.1.0-py27_0.tar.bz2 -o ~/anaconda/conda-bld/*

The directory structure here should be altered for the name of your base conda directory, OS platform and package tarball, of course.

Uploading your package

Additional information on the anaconda cloud can be found here.

To begin uploading your packages, create an account for the anaconda cloud if you have not already done so. You will also need to install the anaconda client:

conda install anaconda-client

Then log in with your account info:

anaconda login

Finally to upload our OSX package as per our example, use:

anaconda upload ~/anaconda/conda-bld/osx-64/ocw-1.1.0-py27_0.tar.bz2

Where again the location should point to your desired package tarball and platform. You should repeat this process for each platform.

Sharing your updated recipe files

Changes to the recipe files should be committed to our git repo with each release, especially if dependencies are changed.


  • No labels