Pushing Impala docs to impala.apache.org

The purpose of this document is to describe how to push rendered documentation to https://impala.apache.org/.

1. Make sure the documentation you want to push a rendered version of is already present in some commit in the ASF git repo at https://gitbox.apache.org/repos/asf/impala.git.

   1. Run push_to_asf.py to push any unpushed commits from Gerrit to ASF.

2. Open a terminal.

3. Every commit in git has a commit hash. You will be pushing the docs from a specific commit hash that you decide upon at this point in the process. Pick it as follows:

1. Go to https://gitbox.apache.org/repos/asf/impala.git.

2. There is a section called shortlog. In shortlog, every line corresponds to a commit. Each line contains a date, an author name, the first line of the commit, and then four links to commit | commitdiff | tree | snapshot.

3. Look for the commit you want to use in the shortlog section.

4. If it is not there, click on the ... (ellipses) at the bottom of the shortlog section. This will take you to a longer list of commits in the same format. Look for the commit you want in that list.

5. If you don't see it there, click the next button at the bottom.

6. The above procedure only helps you find commits that are on the "master" branch. If you know your commit is on a different branch and not on master, you can search there. This is unlikely, though as most commits are to master.

7. To search a given branch, go back to https://gitbox.apache.org/repos/asf/impala.git. There is a section called heads at the bottom of the page. Find the branch name you want to look inside of and click on it. This gives you a shortlog with the next links on the bottom of each page. You can now search for the commit just like you did on master.

8. Once you know which commit you are interested in, click on the blue link that is the first line of the commit. This takes you to a more detailed commit page. It will include a section with the commit's metadata, such as author and date. In that section there is a line starting with "commit" and ending with 40 random-looking hexadecimal characters. That is called the commit hash.

9. In your terminal, type "COMMIT_HASH=". Then paste the commit hash, and then hit "Enter". File away this COMMIT_HASH value later, either by not killing the terminal window, or copy/pasting it into a note elsewhere. You'll need it near the end of this procedure!

4. Switch to the directory you use to interact with Gerrit and Apache.

5. Enter the following sequence of commands:
   

   git fetch https://gitbox.apache.org/repos/asf/impala.git
   git fetch https://gitbox.apache.org/repos/asf/impala.git $COMMIT_HASH
   git checkout FETCH_HEAD
   cd docs

6. Next, you make sure that the build directory contains nothing but a clean copy of the latest generated output. (In the course of other doc-related work, you might have made temporary copies of other HTML or PDF files.) Enter this sequence of commands:

   cd docs
   rm -rf build
   make
   cd ..

7. Copy the build directory to a temporary space:
   

   DOCS_STASH=$(mktemp -d)
   cp -r docs/build ${DOCS_STASH}
   

8. A git remote is a git repository on another computer that you can communicate with. Look at the git remotes that you have in this directory by typing "git remote -v" and hitting "Enter". For me, this looks like:

1. $ git remote -v
   apache  https://gitbox.apache.org/repos/asf/impala.git (fetch)
   apache  https://gitbox.org/repos/asf/impala.git (push)
   gerrit-asf      ssh://jbapple-cloudera@gerrit.cloudera.org:29418/Impala-ASF (fetch)
   gerrit-asf      ssh://jbapple-cloudera@gerrit.cloudera.org:29418/Impala-ASF (push)
   ...

2. In that example, I used the command "git remote -v" to get a list of the remotes I had and their URLs. The user input is on the line that starts with a dollar sign. Do not type the dollar sign; it is just printed there by the terminal to indicate that it is ready for your input. The output from the git command is in the lines below that. The ellipses at the end are some extra info that I am leaving out because it is not relevant for this document.

3. Each remote is listed with a name, a URL, and then either "(fetch)" or "(push)".

9. Identify the name of the git remote that you use to correspond with Gerrit. For me, this is "gerrit-asf". For you, this might be "asf-gerrit".

1. Type "GERRIT_REMOTE_NAME=". Then type the name of the Gerrit remote. Hit enter.

10. Enter this sequence of commands:
    

    git fetch $GERRIT_REMOTE_NAME
    git checkout $GERRIT_REMOTE_NAME/asf-site

11. Think up a name for the branch you are about to create. This is only local, so make it something you can remember but don't worry about other people remembering it.

    1. If there is any output, think up a new name and repeat.

1. Make sure it is a new name: type "git branch | grep ". Type the branch name. Hit enter.

2. Once you have a name, type "DOCS_PUSH_BRANCH_NAME=". Type the branch name. Hit enter.

12. Type "git checkout -b $DOCS_PUSH_BRANCH_NAME" and hit enter.

13. Edit impala-docs.html to point to the newest release:

    1. Update the Latest Releases section to refer to the new version of Impala. For example, if it says Impala 3.2 is the current version, change this to Impala 3.3, assuming you're uploading the docs for 3.3.

    2. The latest release should be listed in the Latest Releases section.
    3. Under Older Releases, add a section for the PDF you just renamed.

14. Copy the docs:

    1. Type: 

       cp -r ${DOCS_STASH}/build/* docs/build/

15. Rename the current release's PDF.
    For example, type: 

    mv docs/build/impala.pdf docs/build/impala-3.3.pdf

16. These 2 files are not needed for the final rendered docs. Delete them in the docs/build directory. Type:

    rm docs/build/html/shared/ImpalaVariables.html
    rm docs/build/html/shared/impala_common.html

17. Type:

    git add docs/build

18. Now contribute to Gerrit like you usually would:

1. Type: 

   git commit -a

2. Write a good commit message. Please follow the guidelines for commit message format. See Contributing to Impala. The commit message should also include the value of COMMIT_HASH you saved near the beginning of this procedure. It may seem strange to include a commit message here, but because we are crossing branches, this is good practice. So the message should include, but not be limited to, something like "Add Impala docs from branch master, commit hash a827e9edc1f93ec0ddb2184133b114182ebb2e02", where a827e9edc1f93ec0ddb2184133b114182ebb2e02 is the hypothetical value of COMMIT_HASH.

3. Type to push to Gerrit for review. 

   git push --no-thin $GERRIT_REMOTE_NAME HEAD:refs/for/asf-site

4. Go to the Gerrit review URL and add reviewers.

19. Review and submit procedures on asf-site are slightly different than on master:

1. Once the patch has been +2ed by a committer, a committer also +1 verifies it.

2. After that, a committer will hit the "Submit" in Gerrit. There is no pre-merge Jenkins job on the asf-master branch.

3. A committer then go to their terminal and run the following to upload your patch to the Apache Impala website.

   git checkout master
   bin/push_to_asf.py