SpamAssassin Infrastructure Notes

This section of the wiki is intended to hold notes on SpamAssassin's infrastructure – mailing lists, svn, where to ask about getting stuff fixed, etc. The intended audience is the SpamAssassin committers. Hopefully most of this page will be simply links to pages in http://www.apache.org/dev/ , rather than duplicating too much of that site.

Reporting Failures

http://issues.apache.org/jira/ is the bug-tracker used to report and track ASF infrastructure issues.

Before reporting an issue, check http://status.apache.org/ which replaced http://monitoring.apache.org/status/ in May 1014.

In some cases, you can also mail infrastructure /at/ apache.org. There are a number of mailing lists for ASF infrastructure management. See http://www.apache.org/dev/infra-mail.html for more information.

Machines we use

http://www.apache.org/dev/machines.html lists the ASF's machines. We use:

minotaur aka people: for publishing releases according to build/README (in svn); spamassassin.apache.org static content - This information is likely out of date but the build/README is accurate.

NOTE: Accounts for minotaur are for committers only – requested via the PMC chair.

SUFFERED CATASTROPHIC FAILURE April 2014: spamassassin.zones.apache.org: a Solaris Zone running on gaea.apache.org, running various services, see SpamAssassinAsfZone

spamassassin-vm.apache.org: A CentOS VM that replaced the Solaris spamassassin.zones.apache.org. This box has the same IP as spamassassin.zones.apache.org as a secondary IP address.

spamassassin2.zones.apache.org: a Solaris Zone running on odyne.apache.org, running various services, see SpamAssassinAsfZone

Accounts for the zone are for committers only – requested via the PMC.

issues.apache.org: ASF bugzilla

Creating accounts, granting karma, etc

  • Account creation is detailed in the PMC FAQ: http://www.apache.org/dev/pmc.html
  • Account karma can be granted by the PMC Chair: InfraNotes GrantingKarma
  • PMC people: http://www.apache.org/dev/solaris-zones.html explains how to create an acct on the Solaris zones. Here is a quick guide:
  • Create the account with this command:
  • sudo useradd -d /export/home/fred -m -c "Fred Smith" fred
  • Set a password with this command:
  • sudo passwd fred
  • Check for password file format errors with this command:
  • sudo pwck
  • Finally, if needed, add the user to sudoers in /etc/opt/sfw with this command:
  • /opt/sfw/sbin/visudo

SSH Access for SpamAssassin VM Box

Create a user with useradd -c 'Full name of User' username that matches their ASF username.

Create a file with their public key for ssh to /etc/ssh/ssh_keys/<username>.pub

Add the username to the sshusers group with useradd -G sshusers <username>

Things to do if you're a new committer in SpamAssassin

an ASF guide doc: http://www.apache.org/dev/new-committers-guide.html

set up SSH access to minotaur: http://www.apache.org/dev/user-ssh.html

Creating rsync accounts: RsyncConfig

Mailing lists

We have several; not sure what machine they're hosted on. If you're a committer, and feel like it, it'd be great if you could help out with moderation – it's just a matter of bouncing spam that's being sent to the lists, bouncing mails being sent to the announce list, and approving people who aren't spammers for the other lists.

Administration and moderation is discussed in http://www.apache.org/dev/committers.html#mail-moderate ; this seems to be mostly up to date. Here's the EZMLM remote admin manual.

More details are at http://www.apache.org/foundation/mailinglists.html .

Note that in general, if an ASF list is for committers or members only (a few are), you need to use your @apache.org address to subscribe, it seems. This isn't written down anywhere I can find, but seems to be the case.

The ASF run a (very basic) archive of all the lists at http://mail-archives.apache.org/mod_mbox/ . There's also something called eyebrowse; don't use it, it's borked.

The web site

The master copy of the spamassassin.apache.org static content is on minotaur at /www/spamassassin.apache.org . It's built with webmake, which is a simple perl website building tool. The source file is called 'main.wmk'. To run webmake on minotaur, read build/README in svn, that tells you how to set up your path to use the installed copy of webmake in jm's home dir.

Ensure your umask is set to 002 before writing to files here – this is VERY important! We don't have root here, and screwups have to be fixed by someone on the infrastructure team who does.

Note that the static content is hosted in SVN at https://svn.apache.org/repos/asf/spamassassin/site/ . If you make any changes on minotaur, be sure to check them in – you may have to copy the changed files from the checkout there into your own checkout, btw, to be able to authenticate correctly to the svn server; or better, do the change in your own checkout first, and simply 'svn up' on minotaur.

Web site visibility

See http://www.apache.org/dev/project-site.html for the official doco.

Our spamassassin.apache.org web site is configured so it displays static pages committed to our svn repo in directory "site". Commits made to there will be served immediately. Remember to refresh your browser cache to see a recent change.

As an alternative location, you also have a public HTML dir on minotaur – it's at http://people.apache.org/~yourusername/ , e.g. http://people.apache.org/~jm/ . This is not mirrored. Files placed in minotaur:~/public_html will be visible immediately. This is good for publishing prereleases and alphas.

The ASF download mirrors

The download mirrors are used specifically to mirror the /dist/ portion of the website; there's not much need to discuss that here, as there's very strict rules on what files can be published there, and their layout, and it's covered in build/README.

It is worth noting in passing though that despite what it says in http://www.apache.org/dev/mirrors.html and http://www.apache.org/dev/mirror-guide-bodewig.html , symbolic links will NOT work in that part of the site on certain mirrors! This has bitten us before. see http://issues.apache.org/jira/browse/INFRA-184

Your apache.org email address

You have a .forward file as discussed at http://www.apache.org/dev/services.html . SSH in and set it up!

Using the Zone

spamassassin.zones.apache.org and spamassassin2.zones.apache.org: see http://www.apache.org/dev/solaris-zones.html . There's some more info at SpamAssassinAsfZone.

/etc/rc3.d/* output goes to /var/svc/log/milestone-multi-user-server:default.log ; rc2.d to milestone-multi-user:default.log .

This mystery process:

 0 S noaccess 10382  9927   0  40 20        ?  45618        ? 10:49:07 ?           0:35 /usr/java/bin/java -server -Xmx128m

is the "Java web console". I've disabled it – and a batch of other unused services – using:

sudo svcadm -v disable webconsole
sudo svcadm -v disable svc:/application/opengl/ogl-select
sudo svcadm -v disable svc:/application/font/stfsloader
sudo svcadm -v disable svc:/application/x11/xfs
sudo svcadm -v disable svc:/network/finger
sudo svcadm -v disable svc:/network/ftp
sudo svcadm -v disable svc:/network/rpc/cde-calendar-manager
sudo svcadm -v disable svc:/network/rpc/cde-ttdbserver
sudo svcadm -v disable svc:/network/telnet
sudo svcadm -v disable svc:/network/cde-spc
sudo svcadm -v disable svc:/network/rpc-100235_1/rpc_ticotsord
sudo svcadm -v disable svc:/application/management/seaport
sudo svcadm -v disable svc:/application/management/snmpdx
sudo svcadm -v disable svc:/application/cde-printinfo
sudo svcadm -v disable svc:/application/font/fc-cache

BuildBot

see ContinuousTesting. when creating slaves on the Solaris zone, you cannot use "buildbot slave" to create it due to a bug in use of ptys; instead:

PASSWORD=slavepassword
NAME=slavename
mkdir /home/bbmass/slaves/$NAME
chdir /home/bbmass/slaves/$NAME
mktap buildbot slave --basedir /home/buildbot/slaves/$NAME \
         --master buildbot.spamassassin.org:9989 --name $NAME \
         --passwd $PASSWORD --usepty=0

RuleQA / Nightly Masschecks

The RuleQA process is running on spamassassin2.zones.apache.org as a daemon named "freqsd".

bash-3.00$ ptree
...
14136 sh -c ./build/automc/freqsd -pidfile /export/home/automc/freqsd/pid
  14137 /local/perl586/bin/perl ./build/automc/freqsd -pidfile /export/home/automc/freq
    14138 /local/perl586/bin/perl ./build/automc/freqsd -pidfile /export/home/automc/freq
      10711 sh -c cd masses ; ./rule-qa/automc/gen_info_xml
        10712 /local/perl586/bin/perl -w ./rule-qa/automc/gen_info_xml
    29465 sh -c cd masses/rule-qa ; ./reports-from-logs --override='output_classes=OVERLA
      29466 /local/perl586/bin/perl -w ./reports-from-logs --override=output_classes=OVERLA

If this daemon is not running (i.e. if the results reported at http://ruleqa.spamassassin.org/ are more than a day old) then it should be restarted:

# Update from SVN, just to make sure everything's current:

cd /export/home/svn-trunk
sudo -u automc -H svn up

# Restart the daemon

sudo -H /etc/init.d/freqsd restart

Bugzilla

Is now at the ASF at http://issues.apache.org/SpamAssassin/ .

Updating the SVN Certs

see SvnCertUpdate.

  • No labels