Bugzilla Management

Basics

Hosts

The Bugzilla instances and associated database instances are located at Hetzner Online. The hosts are:

1. bz.apache.org - the actual BZ instances
2. mysql1.us-mid.priv.apache.org - primary db
3. mysql2.us-mid.priv.apache.org - backup db

Layout

There are four separate Bugzilla instances:

Starting and Stopping

All four BZ instances currently run on the same apache24 install as vhosts. To restart:

service apache2 restart

To refresh:

service apache2 reload

Installation

Installation of BZ is done all through puppet.

The only outside manual steps are:

1. Adding the information to create the mysql database in puppet
2. Building and updating the deb package (see Upgrading section below)
3. Uploading the deb package to Artifactory

Note: as of June, 2021 Bintray is deprecated and Artifactory is in its place.

Everything else is contained in the bugzilla puppet module and the bz-he-de.apache.org.[e]yaml files in the puppet 6 repo.

There is a provision in the bugzilla module to install perl libs using CPAN, but this is highly discouraged in favor of native packages and/or using fpm. This is because the native packages will get security fixes whereas modules installed via CPAN will need to be manually updated. The current exceptions are:

1. PatchReader (not provided by Ubuntu repos)
2. DateTime::TimeZone (version required by BZ 5.0.x not recent enough in Ubuntu Trusty)
3. Email::Sender (version required by BZ 5.0.x not recent enough in Ubuntu Trusty)
4. Text::Wrap (version in Ubuntu Trusty has a bug that impacts Bugzilla)

Bugzilla is built manually from the ASF's locally modified version kept in svn, but installed using Puppet. For details of the process of updating and installing that version see the upgrading section below.

The OOo BZ instance required the following additional mysql configuration over and above that defined in the docs: In /etc/mysql/my.cnf on mysql{1,2}-us-mid.priv.apache.org (and is set already using Puppet)

max_allowed_packet=16M

Upgrading

1) Check out the relevant parts of the svn tree for Bugzilla. Assumes an svn 1.5 or later client

$ svn co --depth immediates https://svn.apache.org/repos/infra/infrastructure/ asf-infra
$ cd asf-infra
$ svn up --set-depth infinity bugzilla
$ svn up --set-depth immediates vendor
$ cd vendor
$ svn up --set-depth immediates bugzilla
$ cd bugzilla
$ svn up --set-depth infinity current

2) Download and extract the new Bugzilla version.

$ cd ~/downloads
$ wget http://ftp.mozilla.org/pub/mozilla.org/webtools/bugzilla-<version>.tar.gz
$ tar xfz bugzilla-<version>.tar.gz

3) Import the new version and drop into into infrastructure svn.

$ <path_to_checkout>/bugzilla/tools/svn_load_dirs.pl -t <version>     \
    -p <path_to_checkout>/bugzilla/local/svn_load_dirs_property_table \
    [-wc <path_to_checkout>/vendor/bugzilla/current]                  \
    https://svn.apache.org/repos/infra/infrastructure/vendor/bugzilla \
    current                                                           \
    <path_to>bugzilla-<version>

4) Merge the differences between the current and previous version into our local version.

$ svn merge \
   https://svn.apache.org/repos/infra/infrastructure/vendor/bugzilla/<previous-version> \
   https://svn.apache.org/repos/infra/infrastructure/vendor/bugzilla/current \
   bugzilla

5) Resolve any conflicts that the merge came up with.

6) Then, cd to bugzilla/template/en/custom and find all *.html.tmpl files. This is our custom content. Carry over any changes in the corresponding template/en/default hierarchy to their custom counterparts.

$ pwd
<...>/infrastructure/bugzilla/bugzilla/template/en/custom/global
$ svn diff ../../default/global/code-error.html.tmpl \
    | patch -p0 code-error.html.tmpl

Resolve any failed patch chunks.

Do this for every customized template.

7) Once you're done with this, commit the whole infrastructure/bugzilla/bugzilla hierarchy.

$ pwd
<...>/infrastructure/bugzilla/bugzilla
$ svn ci -m "Upgrade to Bugzilla Release xx"
...

Congratulations! You now have a new version drop of Bugzilla in svn, complete with our local modifications!

Steps 4 to 7 need to be repeated for each of the 4 Bugzilla instances.

8) Build the updated package: run the build script from infrastructure/bugzilla/tools/fpm-build:

$ ./buildbz.sh  -v 4.4.6 -n asf -d "ASF Bugzilla"
# where -v is the version, -n is the tree (asf, sa, ooo, nb), and -d is the description

Assuming no errors, you should have a .deb file looking something like:

bugzilla-$tree_$version-$date_amd64.deb

Repeat this for all four Bugzilla trees.

9) Upload the .deb to Artifactory

1. Add new versions
2. Upload files
3. arch=amd64, distribution=bionic,focal, component=main

10) Publish the files in Artifactory

11) Open up ASF Bugzilla in your browser, and run a sanity check.

12) Then go to the Parameters page, find the shutdownhtml parameter and enter a meaningful message like "ASF Bugzilla is currently down for a version upgrade. Please check back in a few minutes. We apologize for the inconvenience. " Click 'Submit changes' at the bottom. Bugzilla is now unavailable to users and can be safely upgraded.

13) Before pushing the node manifest change, copy Bugzilla's data folder to /tmp/ (This step only required until a bug in fpm related to argument parsing with pre/post install scripts is fixed)

# mkdir /tmp/bugzilla-$tree
# rsync -a /var/www/bugzilla-$tree/data /tmp/bugzilla-$tree/

14) Commit node's manifest to git, and then run puppet on the node.

# puppet agent -t

15) Go back to the parameters page, remove the shutdownhtml prose and submit.

16) Do another sanity check from the link bar at the bottom of any Bugzilla page.

17) Use the <pathtosvn_checkout>/bugzilla/tools/test-bugzilla-rpc.sh script to check that RPC still works.

Repeat steps 11 to 17 for each of the four Bugzilla instances.