Start here for your first website build. It is helpful to clear out any issues with a simple website build with one version before you try to tackle a more complicated website build with multiple versions.

The following describes how the website and docs are generated. You have two entry points that do the same thing:

  1. make docs (from the mxnet root)
  2. make html (from the docs folder)

Sphinx is the primary engine that controls the site template, calling external docs generators for certain APIs, and normalizing the content for the website. The output is a full website that is static html. You can find it in the /docs/_build/html folder. 

Figure 1. Diagram of the website build flow process using Sphinx and Sphinx extensions

Walkthrough of the Build Steps

  1. To start from scratch, fire up a Ubuntu 16.04 EC2 instance with at least 20 GB of space. More is better if you're doing a lot of testing. A c5d.4xlarge with 100GB SSD is recommended. If you have recently built the docs and you're looking to understand the new pipeline, a.k.a. Fawkes, and its features, skip ahead to Step 2. If you haven't already, you need to fork the MXNet repo. Go to https://github.com/apache/incubator-mxnet and click the Fork button in the upper right.
    1. From your instance, clone the MXNet repo with:

      git clone --recursive https://github.com/<your-fork>/incubator-mxnet

      Or... if you just want to build what's in the main repo, you can clone that instead:

      git clone --recursive https://github.com/apache/incubator-mxnet


    2. Then run the setup script:

      cd incubator-mxnet/docs/build_version_doc/ && ./setup_docs_ubuntu.sh

    3. You will see instructions for turning on mod rewrite. This will activate redirects, and this step is optional.
    4. You will need to open up HTTP (port 80) on the server for all traffic (since you want to share the preview link, right?). Do this from the EC2 console security group settings. The firewall on the server is already opened.

  2. If you have a working setup with all of the dependencies, start with a clone or fork of MXNet. From here let's assume you are using your own fork.
  3. Checkout the branch you want to preview on the site.
  4. Navigate to the `docs` directory.
  5. Open `settings.ini` for editing.
  6. Review each section. 
    1. It is broken up by version showing both the version's tag and branch. Branches have a "v" before the version number.
    2. Each section lists the language APIs and which one will be run for each version. 
  7. For a development fork and branch, the "document_sets_default" section will apply. Turn on the APIs you want to run. The Python API runs no matter what. Turn all the others off. For your final test it is a good idea to turn everything back on. For example, this is a configuration to run Python plus the Scala docs:
    [document_sets_default] 
    clojure_docs = 0
    doxygen_docs = 0 
    r_docs = 0 
    scala_docs = 1


  8. Save the settings file.
  9. Run `make html`. You can also use `make html USE_OPENMP=1` which seems to make it a bit faster on Ubuntu. 
    1. Note: This will not work by default on macOS, so just do `make html` or go through the setup to enable OpenMP on macOS.
  10. The output will be in the `docs/_build/html` folder. Clean out the Apache web server folder (remnant files can mess up your testing and validation), then copy the new website content to it with:
    sudo rm -rf /var/www/html/* && sudo cp -a ~/incubator-mxnet/docs/_build/html/* /var/www/html/

  11. Find the public IP for your instance on your EC2 console and copy it. If you didn't open it up for HTTP yet, now's the time. You can view the site in your browser by the visiting the IP you copied.

How to build R Docs

At this point you may not have the required dependencies to generate R docs. On Ubuntu these would be:

sudo apt-get install \ 
    texinfo \ 
    texlive \ 
    texlive-fonts-extra 

You will also need to enable R docs in settings.ini. For your own dev branch you would change the default section like so:

[document_sets_default] 
clojure_docs = 0 
doxygen_docs = 0 
r_docs = 1 
scala_docs = 0

The output is a .pdf file that can be found at docs/api/r/mxnet-r-reference-manual.pdf.

How to test the settings file

Assuming you have the setup defined previously, and say you're testing the settings file and you want to make sure it is running the right API docs for that version:

  1. Use the `BUILD_VER` flag and pass in the value of the version you want to test. By default, by not specifying BUILD_VER, you're running your current branch, therefore the `document_sets_default` settings in `settings.ini`. For example, to specifically test the build steps for version 1.0.0:
    make html USE_OPENMP=1 BUILD_VER=1.0.0

Troubleshooting

Sometimes the submodules get updated in master and you have to specifically update them yourself. If you see a build error that crashes on mshadow or another submodule's build you likely need to run the following command in your MXNet repo:

git submodule update --init --recursive

Another more rare occurrence is someone did something fairly major and you get a build error in MXNet like "no rule to make target...". Try running the following in the MXNet repo root:

make clean

For other issues, please open a GitHub issue, or try one of the other communication methods to seek help.

Next up

Production website build pipeline - shows you how to build the full site with the "Versions" dropdown.