You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 3 Next »

Why does the AWL sometimes assign scores the "wrong way"?

It's a common misconception that the AWL should always contribute a positive adjustment to the score of spam, and a negative one to nonspam. However, that's not the case, and it's not a bug or misconfiguration of SA. In fact, the AWL wouldn't work as well as it does if it didn't assign scores in the "wrong way" sometimes.

Fundamentally, the AWL is a score averaging system. You can read the "How does It Work" section in AutoWhitelist for a more detailed explanation, but for now it should be sufficient to know that it works by keeping track of the historical average score for a given sender, and adjusting the score of the message towards their past average.

Now, given that it's an averaging system, it would be impossible for it to always assign negative scores to nonspam messages, unless the pre-AWL scores of those messages were always above the average. In the long run, that would only happen if the email scores from that sender were increasing, meaning that the average from that sender was constantly increasing.

Really the fact that the AWL sometimes assigns scores the "wrong way" for the type of mail isn't a problem, unless the AWL pushes the email across your tagging threshold in the wrong direction, which means that the average for that sender is bad and needs to be fixed. (The --add-to-whitelist and --add-to-blacklist command line switches are currently designed for this use, but check with the man page.)

For example let's say my friend sends me mostly emails in rough ballpark of score=1.0, but one this day sends an email that happens to trip off a negative scoring rule or two and winds up scoring a -4 before the AWL is factored in. The AWL will look at his past average of 1.0 and the current email of -4, and it will average them. The average of these scores is -1.5, so the AWL will add 2.5 points to the message. However, the email is still correctly categorized as not being spam.

However, if that same friend instead sent me an email that had some racy jokes in it, and scored a 7, the AWL would again average 1.0 and 7. The resulting average is 4, so the AWL would knock 3 points off the score in this case.

The same basic effects also happen with spam, where a past average of 10, and a current score of 20 can result in the AWL knocking 5 points off the score, resulting in a final score of 15.

As you can see from these examples, the AWL operates by averaging out the "score spikes" between different emails. By the very nature of averages, this means that it will push the high points down, and the low points up, but it will always push them towards the average for that sender. As long as the average for that sender is on the right side of the spam/nonspam fence, it will do it's job nicely.

(by MattKettler)

  • No labels