This Confluence has been LDAP enabled, if you are an ASF Committer, please use your LDAP Credentials to login. Any problems file an INFRA jira ticket please.

Page tree
Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 12 Next »

Note: This page is under construction and shouldn't be considered "complete". If there are any questions, etc, please feel free to contact the users at spamassassin.apache.org list for more information.


Notes / FAQ

  • Currently, for 3.1.1+ and 3.2.0, to use any channel for updates requires that updates.spamassassin.org also be used. This is because once the update directory exists, the SpamAssassin modules expect to find all rules in that directory.
    Can plugins be distributed via updates/channels?


    From a technical standpoint, updates can contain plugins. However, the default channel, updates.spamassassin.org, will not publish plugins using this method at this time since people are likely not ready to accept automatic code updates.


    After sa-update completes, do I have to move the files somewhere for them to be used?


    No. By default, sa-update and the SpamAssassin modules use the same location for updates. This means that after a successful update run, the new rules are available for use. As usual, if using spamd, a restart is required for the new rules to be read in and enabled.


    Should I use "--updatedir" to put updates in the default rules directory (ie: /usr/share/spamassassin), or the site rules directory (ie: /etc/mail/spamassassin)?


    No! Those two directories have specific uses, and attempting to install updates to that directory will likely cripple your SpamAssassin installation. This is because when an update is installed, all previous files in the directory are removed first.


    If I use "--updatedir" to install an update in a subdirectory of my site rules directory (ie: /etc/mail/spamassassin), a number of my local settings (typically set via local.cf) no longer function. Why?


    This can happen if the update channel creates a channel file with a name lexically after the local settings, such as "updates_spamassassin_org.cf". For more information, read the "Using sa-update" section below, and also refer to the man page for "spamassassin".


    What do I need to do with-respect-to sa-update after a SpamAssassin upgrade?


    Whenever you upgrade the version of SpamAssassin that's installed, the directory where the updates are expected to be changes (it's based on the version). So whenever you upgrade, you will want to run sa-update for all of the channels that you want to have installed.


    After upgrading SpamAssassin several times, there are a number of directories .../3.001001, .../3.001002, .../3.001003, etc. What should I do with them?


    You definitely want to keep the latest version that matches the version of SpamAssassin that you have installed. However, the older versions can be removed whenever you like.


    After sa-update was used, the report contact setting becomes @@.


    This was fixed in 3.1.4. This issue was tracked in [http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4862 bug 4862].


For more information about how to use sa-update and how it works, please read below.


Using sa-update

What is sa-update?

The goal of sa-update is to download new configuration files (rules, scores, etc,) so that SpamAssassin will use them to better catch spam and/or to avoid catching ham messages. The main reason to use sa-update is that the old method of disseminating rules, releasing a new version SpamAssassin, is a lengthy process that can take many months. Spam is rapidly changing, and new rules are often written in response. With sa-update, those rules can quickly (potentially within minutes) be distributed and the new spam caught.

Simply put, sa-update allows rules to be distributed as they are developed, while full SpamAssassin releases can focus on bug fixes and new features.

Channels

Simply put, channels are locations where sa-update can download rule and configuration files. By default, sa-update will use the updates.spamassassin.org channel to receive official updates from the SpamAssassin project, but anyone can create a channel and publish updates. By default, sa-update (and spamassassin) expect to find updates in the /var/lib/spamassassin/<spamassassin version> directory, which will have each channel in its own directory underneath. Each channel will also have a channel cf file and (optionally) a channel pre file to load the update's configurations in the update's parent directory. For example:

/var/lib/spamassassin
`-- 3.001004
    |-- updates_spamassassin_org
    `-- updates_spamassassin_org.cf

Shows the channel updates.spamassassin.org available for SpamAssassin 3.1.4, underneath the /var/lib/spamassassin directory.

For more information about what makes up a channel and how it all works together, please see the Publishing channels for sa-update section below.

NOTE: Once the /var/lib/spamassassin/<spamassassin version> directory exists, spamassassin expects to find all rules underneath that directory, so make sure that the first time you run sa-update it completes successfully (see below for information about running in debug mode).

sa-update commandline

sa-update has several parameters that can be passed via the commandline.

Channel / Location Related

--updatedir <path>:: sa-update by default places updates in the /var/lib/spamassassin/<spamassassin version> directory. If updates should go into a different directory, specify it via this option.

--channel <channel>:: sa-update by default only uses the updates.spamassassin.org channel for updates. If other channels should be used, specify it with this option. For multiple channels, specify this option multiple times. ie:

 sa-update --channel updates.spamassassin.org --channel saupdates.example.com 

--channelfile <file>:: If multiple channels are going to be used at once, it may be easier to write the channels to a file, and then use this option to point to that file. ie:

$ rm -f channels ; touch channels
$ echo updates.spamassassin.org >> channels
$ echo saupdates.example.com >> channels
$ sa-update --channelfile channels

GPG Related

--(no)gpg:: By default, sa-update will require the use of GPG signatures to verify that downloaded updates are in fact legitimate. However, there may be channels which do not publish GPG signatures, or the system may not have GPG installed. In these situations, specify the

 --nogpg 

option to disable the use of GPG. Note: By using the --gpgkey or --gpgkeyfile options as shown below, --gpg is automatically enabled.

--gpghomedir <path>:: sa-update tries to keep its keys separate from the user's keys by using a different directory for the keyrings (passed to gpg via its --homedir option). By default, the location is /etc/mail/spamassassin/sa-update-keys. If a different location is desired, use this option to specify it.

--gpgkey <key id>:: Specify which GPG key ids should be trusted to sign update packages. If there are multiple keys, use this option multiple times to list them. Generally it's safer to specify the whole key fingerprint, but it is more common to see simply the last 8 hex digits used. ie:

 sa-update --gpgkey 26C900A46DD40CD5AD24F6D7DEE01987265FA05B --gpgkey 5244EC45 

--gpgkeyfile <file>:: Similar to channelfile, if there are multiple keys to be trusted, it may be easier to specify them in a file and then use this option to point sa-update at the file. ie:

$ rm -f gpgkeys ; touch gpgkeys
$ echo 26C900A46DD40CD5AD24F6D7DEE01987265FA05B >> gpgkeys
$ echo 5244EC45 >> gpgkeys
$ sa-update --gpgkeyfile gpgkeys

Other

-D, --debug [area=n,...]:: Show debugging information. This can be useful just to see what sa-update is doing, but is also useful to debug problems, etc. This option takes the same optional parameter (areas) as the other standard SpamAssassin tools.

-V, --version:: Display which version of sa-update is installed. sa-update is versioned by Subversion revision number as opposed to being tied to a specific SpamAssassin version.

-h, --help:: Print usage message.

More information is available via the POD/man page for sa-update.

Examples

sa-update && service spamassassin restart

This is potentially the most simple example of using sa-update. It will see if an update is available, and download the update and lint-test it as necessary. If there was an update and no problems were detected, sa-update returns 0 and the

 service spamassassin restart 

command is run (used on Linux machines to restart the spamassassin service (spamd)).

sa-update -D

If there is a problem with doing updates, or just some curiousity about what is happening, run sa-update in debug mode and it will show you what it is doing step-by-step.

sa-update -D --updatedir /tmp/updates

To perform updates without attempting to put the files into the system-wide location (either for users or to test, etc,) use the --updatedir option to aim at a different directory for which to put the updates.

Installed Updates

When updates are downloaded, they are put into a directory under the local state dir (default /var/lib/spamassassin/<spamassassin version>) similar to:

/var/lib/spamassassin
`-- 3.001004
    |-- updates_spamassassin_org
    `-- updates_spamassassin_org.cf

The files from the update go into updates_spamassassin_org, and the *.cf files are then included by updates_spamassassin_org.cf, which also keeps track of what update version is installed. Therefore, if it is desired to change the update directory, the .cf and the update directory will exist there.


Publishing channels for sa-update

Channels are fairly simple to setup and use. Simply put, updates are files contained in a standard tar.gz archive, distributed via HTTP. To allow for frequent update checks from clients, a lightweight method (DNS queries) is used to specify what update version is the latest for any given version of SpamAssassin. sa-update also uses a mirror file which lists all of the URLs where the update can be downloaded from, optionally including weights for different mirrors.

How does it work?

When looking for an update, sa-update reverses the version and makes a DNS TXT query for <version>.<channel>. ie: Running 3.1.1's sa-update, the default updates.spamassassin.org channel causes a DNS lookup for 1.1.3.updates.spamassassin.org. The query is for a TXT record containing the update number, which should be a monotonically increasing value. Assuming an update is necessary, sa-update will then read the MIRRORED.BY file (downloading it first if necessary from the URL found in a DNS TXT record mirrors.<channel>).

Summary

DNS Records:

  • mirrors.<channel> TXT "http://URL/TO/MIRRORS/FILE"
     mirrors.updates.spamassassin.org TXT "http://spamassassin.apache.org/updates/MIRRORED.BY" 
  • <version>.<channel> TXT "UPDATE_NUMBER"
     *.1.3.updates.spamassassin.org TXT "386156" 

Mirrors file:

  • List of URLs which contain the updates
     http://buildbot.spamassassin.org.nyud.net:8090/updatestage/ 

Update file:

  • Only files (no directories) contained in a tar.gz archive file named UPDATE_NUMBER.tar.gz
  • A sha1sum file named UPDATE_NUMBER.tar.gz.sha1 with the output of
     sha1sum UPDATE_NUMBER.tar.gz 
    .
  • Recommended, but optional, a detached GPG signature for the update named UPDATE_NUMBER.tar.gz.asc via something like
     gpg -bas UPDATE_NUMBER.tar.gz 

Examples

Here is a short example of how an update for SpamAssassin 3.1.x would be published. By convention, we use the svn version of the directory as the update number.

$ svnversion
386156
$ ls -al
total 528
drwxr-xr-x  3 felicity fame  4096 Mar 14 13:10 .
drwxr-xr-x  4 felicity fame    27 Feb 19 14:55 ..
-rw-r--r--  1 felicity fame  5479 Feb 19 14:55 10_misc.cf
-rw-r--r--  1 felicity fame  8112 Mar 13 02:49 20_advance_fee.cf
-rw-r--r--  1 felicity fame  1602 Feb 19 14:55 20_anti_ratware.cf
-rw-r--r--  1 felicity fame  6690 Mar 13 02:49 20_body_tests.cf
-rw-r--r--  1 felicity fame  1534 Mar 13 02:49 20_compensate.cf
[...]
-rw-r--r--  1 felicity fame 33266 Mar 11 18:13 50_scores.cf
-rw-r--r--  1 felicity fame  8564 Mar 14 13:10 80_additional.cf
$ tar -cf 386156.tar *.cf
$ ls -la 386156.tar 
-rw-r--r--  1 felicity fame 471040 Mar 15 15:37 386156.tar
$ gzip -9v 386156.tar 
386156.tar:      74.1% -- replaced with 386156.tar.gz
$ ls -la 386156.tar.gz 
-rw-r--r--  1 felicity fame 122206 Mar 15 15:37 386156.tar.gz
$ mv 386156.tar.gz /www/spamassassin/updates
$ vi /var/named/updates.spamassassin.org
[increase serial number, add in new update line '*.1.3 TXT "386156"']
$ rndc reload
  • What if I want to publish updates for some versions but not others?

The answer is pretty simple and comes down to DNS records. As long as the reverse version DNS request does not return a TXT record, sa-update will consider there to be no updates available. ie: if we want to publish update 386156 for SpamAssassin 3.1.x, but not 3.1.0, we could use the following records:

0.1.3 A   127.0.0.1
*.1.3 TXT "386156"

When v3.1.0's sa-update looks for an update, it gets no TXT response (having an existing 0.1.3 record overrides the wildcard record), and therefore it sees no updates available. However, when v3.1.1 or above looks for an update, it gets "386156" returned.

SARE Channels

Details of various channels from SARE can be seen at SAREChannels

The Backend

Details of the rule-update generation backend at updates.spamassassin.org can be read at SaUpdateBackend.


TheoVanDinter

  • No labels