Introduction
OCW Installation Instructions
Requirements
- The latest version of conda. The recommended way to obtain this is to install either the Anaconda or Miniconda scientific python distributions. Be sure to allow the installation to update your PATH for you.
Installation Information
In addition to the standard scientific python packages bundled with most anaconda installations, the following packages will be installed:
- requests: Used to make HTTP requests. The OCW UI and Toolkit use this.
- bottle: Simple Python backend webserver used by the OCW UI.
- pydap: A Python library for connecting to OpenDAP servers. The Toolkit uses this to handle OpenDAP connections.
- webtest: A Python library to improve testing of web applications. The OCW UI backend uses this for testing.
- pyesgf: A Python library for downloading files from the Earth System Grid Federation database.
How To
Before invoking conda to install the dependencies, it's always a good idea to ensure that your version of conda is up to date. You can do this with:
conda update conda
Then to install OCW and its dependencies into your conda environment (by default in ~/anaconda or ~/miniconda), you may use:
conda install -c agoodman ocw
Testing
To give yourself some peace of mind that such a simple installation method actually works, it is recommended that you test you installation. The main OCW codebase has some simple examples which may be obtained using:
cd git clone https://github.com/apache/climate.git
If you do not wish to directly use git, you may download and extract a copy of the code directly from our GitHub page. Once you've obtained the code, head on over to the examples and run one of them.
cd climate/examples python simple_model_to_model_bias.py
After the evaluation runs you should find a .png file in the examples directory. Congratulations, your install was successful!
Fetching Updates and changing versions
Keeping OCW up to date is even more simple than the installation itself. Simply use:
conda update -c agoodman ocw
conda update -c agoodman ocw=1.1.0
Creating a clean environment
It should be noted that the above instructions will install OCW and its dependencies into your default conda environment. For those users who wish to perform development and testing in a clean and isolated environment, they should instead perform their installations using:
conda create -c agoodman -n ocw ocw
This will create a new virtual environment named "ocw" with only OCW and its dependencies installed. To activate this environment, use:
source activate ocw
activate ocw
To deactivate the environment, use:
source deactivate
deactivate
See this page for further documentation on virtual conda environments.
Maintaining the conda packages (for advanced users)
The following section contains some useful information for those in the OCW developer community who are interested in maintaining the conda packages.
Adding the OCW channel to your conda configuration
You may have noticed that many of the conda commands entered above include a -c flag. This is because the OCW package as well as some of its dependencies cannot be found in the default conda channels, making it necessary for us to host them on a separate channel. Since adding this information to each command can get cumbersome, we may simply add it to our configuration as follows:
conda config --add channels agoodman
Now you can perform any of the above commands without needing to specify '-c agoodman'.
To upload your packages to the main OCW channel, you will need to obtain a secure token which you may request by sending an email to the dev mailing list (dev@climate.apache.org). If approved, you may update your config with:
conda config --add channels https://conda.anaconda.org/t/<TOKEN>/agoodman
Where <TOKEN> is the token string.
Updating and Building the OCW conda package
It is highly recommended that you read the conda build documentation first.
Edit the recipe configuration file
The main recipes for the OCW package (and the non-default conda dependencies) can be found in the OCW codebase. Navigate to the conda_recipes directory:
cd climate/conda_recipes
Each subdirectory contains the recipes specific to each package, and these too will be necessary to edit, rebuild, and upload to the OCW channel if we wish to keep everything up to date. For now though we will focus our example workflow on how to update the main ocw package itself. Our hypothetical case will involve creating a package for version 1.1.0. Open ocw/meta.yaml and edit the following entries:
# Under package: version: 1.1.0 # Under source: git_rev: 1.1.0
Here of course the version numbers could be set to whatever release you want. Leaving the git_rev entry commented out will fetch the latest code from the git repo rather than a stable release. Avoid doing this unless you are sure the latest revision of the code is about to be released. If needed, you should also add any new dependencies under the run section.
Build and test the package
You should now save your changes and prepare to build the package. If you do not already have it installed, you will first need to obtain the latest version of conda build:
conda update conda conda install conda-build
To build the package, being sure you are in the conda_recipes directory, use:
conda build ocw
If successful, your newly built package will be found in a tarball located in your conda environment's <conda_environment>/conda-bld/<platform> directory. For an anaconda user's default environment on Mac OSX, this would be located in ~/anaconda/conda-bld/osx-64.
Before proceeding, you should (preferably in a clean environment) install and test your newly built package locally:
conda install --use-local ocw
If everything checks out, you are finally ready to move on to the next step.
Uploading packages to the OCW conda channel
Convert the package to other platforms
Having successfully built a package on your system, you should first add versions for all available platforms. The Mac OSX anaconda user in our example would do this with:
conda convert --platform all ~/anaconda/conda-bld/osx-64/ocw-1.1.0-py27_0.tar.bz2 -o ~/anaconda/conda-bld/*
The directory structure here should be altered for the name of your base conda directory, OS platform and package tarball, of course.
Uploading your package
Additional information on the anaconda cloud can be found here.
To begin uploading your packages, create an account for the anaconda cloud if you have not already done so. You will also need to install the anaconda client:
conda install anaconda-client
Then log in with your account info:
anaconda login
Finally to upload our OSX package as per our example, use:
anaconda upload ~/anaconda/conda-bld/osx-64/ocw-1.1.0-py27_0.tar.bz2
Where again the location should point to your desired package tarball and platform. You should repeat this process for each platform.
Sharing your update recipe files
Changes to the recipe files should be committed to our git repo with each release, especially if dependencies are changed.