Anti-Spam strategies

[Miles]

I am in the process of reviewing the anti-spam measures on two of my Xserves (one running 10.4.3, the other 10.3.9 for now). I have Spamassassin running on both but I am evaluating their effectiveness as (especially the 10.3 server) they are both letting a fair bit of spam through.

I think that part of the problem with the default setup out of the 10.4 box is that many of the SA tests rely on net access, which doesn’t seem to be enabled. I had to jump through a number of hoops to finally get baysean scanning working on 10.4 and I assume it will continue to get better as it gets trained. I have looked at extra SA .cf files, but I have so far not implemented any. Currently, what would seem to be very obvious spam mails are coming through with scores around 3. I have the server set to tag at 5, but there are so far just a few messages that are actually getting tagged.

I would like to get URI blacklisting working and I am looking at greylisting as well.

I would like to know what others are doing and what sort of success you are having fighting the spammers.

Thanks,
Miles

[thinbits]

SA is not working for you becuase the Bayes filter is busted in 10.4 (assuming you’ve done all the required training of the filter).

See this thread for details on fixing it:

here

[Miles]

Hi there,

I had seen that thread and Bayes was working when I first posted. It is by now fairly well trained (about 2000 spam) and not too bad as far as effectiveness goes. It doesn’t seem to give enough points to make the message recognizable as spam though.

There are also a series of tests that I was interested in making work that are dependant on network tests (i.e. looking up stuff on blacklists on the internet etc.) Postfix is already configured to do this (and can use RBLs that are set in the GUI for Mac OS X Server), but Spamassassin is not and relies on other mechanisms in order to access the web.

Here’s a basic step-by-step:
1) enable network tests in amavisd.conf. There are two versions of this file on Mac OS X Server (why? dunno!!) at /etc/amavisd.conf and at /etc/spam/clamav/amavisd.conf. Change this value: $sa_local_tests_only = 0; (default is 1). I changed it in both files.

2) Install Net::DNS. I used CPAN on the command line – ‘sudo cpan -i Net::DNS’ then follow all the prompts answering as appropriate for your setup. You can allow cpan to try and figure out your settings automatically, but generally you would be better served to set the values manually. Allow CPAN to install all dependencies.

3) Make sure that you can do DNS lookups quickly enough. In my case, I had my en0 network interface configured with its own IP as the DNS server i.e.: 11.22.33.44 – this wasn’t fast enough (it might have been slowed down by the firewall? just a guess…) I put 127.0.0.1 as the first DNS server and then it was all good.

Here’s how to test: take a message that you know to be spam and copy it (including full headers cmd-option-U in Mail) into a text file (I used pico, then pasted from Mail) and saved it as testmsg in my home folder. Now you can run spamassassin on it like this from the command line: spamassassin -D < ~/testmsg

As long as the path to the message was OK, you should get a great deal of info regarding what modules SA is using, and details on how it scores your test message. The important bit to check is that DNS works like this:

debug: is Net::DNS::Resolver available? yes
debug: Net::DNS version: 0.55
debug: trying (3) kernel.org…
debug: looking up NS for ‘kernel.org’
debug: NS lookup of kernel.org succeeded => Dns available (set dns_available to hardcode)
debug: is DNS available? 1

If the DNS fails, you will get a message like this instead:

debug: Net::DNS version: 0.55
debug: trying (3) linux.org…
debug: looking up NS for ‘linux.org’
debug: NS lookup of linux.org failed horribly => Perhaps your resolv.conf isn’t pointing at a valid server?
debug: All NS queries failed => DNS unavailable (set dns_available to override)
debug: is DNS available? 0

I’m not sure how to hardcode dns_available so that amavisd calls SA with this switch, but for me, setting 127.0.0.1 as the DNS server did the trick.

Also, when I was trying to figue this all out, the good people on the SA-users list pointed out that the config file for spamassassin has a number of wrong entries in the default OS X install. Run spamassassin –lint to see what’s broken. I think a number of these settings are overridden when amavis calls SA, but it is probably good form to try to fix them in /etc/mail/spamassassin/local.cf.

Here’s my broken settings:

saskatoon:/etc/mail/spamassassin root# spamassassin –lint
config: SpamAssassin failed to parse line, skipping: auto_learn 1
config: SpamAssassin failed to parse line, skipping: safe_reporting 0
config: SpamAssassin failed to parse line, skipping: use_terse_report 0
config: SpamAssassin failed to parse line, skipping: subject_tag *** Warning: Junk Mail ***
config: SpamAssassin failed to parse line, skipping: rewrite_subject 0
lint: 5 issues detected. please rerun with debug enabled for more information.

To fix:
– auto_learn was replaced with bayes_auto_learn in SA 2.50 so just rename that key
– its safe_report, not safe_reporting go ahead and change that
– use_terse_report 0 is obsolete, this is superceded by the report_template commands – just comment it out with a #
– subject_tag and rewrite_subject was replaced with rewrite_header Subject in SA 3.0.0 plus they seem to be controlled by the amavisd.conf file so just comment them out.

Now issuing ‘spamassassin –lint’ should just return you to your command prompt without any output.

There are other SA modules that are not installed/enabled by default in Tiger server that could probably help in controlling the onslaught of spam – pyzor and Razor2 are notable examples (they also show up in the spamassassin debug output) however, I was not able to install them from CPAN, and since this server is in production, I didn’t want to break anything just to get those optional modules. If anyone has instructions on how to install (are they available through fink?), I’d be glad to hear about it. For now, just adding the network tests seems to be pushing most of our spam scores over the threshold I have set at 5 (there were getting around three before – even with strong bayes scores).

There is new project here http://wiki.apache.org/spamassassin/StatsPlugin that will compile stats on spamassassin. I haven’t tried it yet, but it might be interesting to see what your SA is doing.

Hope this helps someone else!

Miles

[TvE]

…/etc/mail/spamassassin/local.cf.

Here’s my broken settings:
…
config: SpamAssassin failed to parse line, skipping: safe_reporting 0
…

To fix:
– its safe_report, not safe_reporting go ahead and change that

Miles

Well no it’s not – its report_safe

[Anonymous]

Hi,
I am still running 10.3.9 but have installed Tiger server at a client’s and was slightly disappointed with the out-of-the box antispam config – my own system lets through a couple of spams a day, purely by using dns blacklists and nothing more! Although Tiger server is a lot more comprehensive in being able to tweak SpamAssassin etc, it also requires way-y-y-y more fiddling to get it right. The DNS blacklists I use are:
dnsbl.sorbs.net
sbl-xbl.spamhaus.org
cbl.abuseat.org
combined.njabl.org
bl.spamcop.net
cn-kr.blackholes.us (China & Korea)
ru.blackholes.us (Russia)
pl.blackholes.us (Poland)

I’ve no idea how expensive these lookups are, but without any further tweaking or experimentation, I am at a totally acceptable level of spamminess. As you can see I have the luxury of never expecting mail from Russia, Korea, China & Poland and this may not be suitable for everyone.
I figure a URI-based blacklist which blocks Chinese-hosted domains would solve the world’s spam problems overnight, any of the top spammers have to be hosted in China. If anyone knows of one, I would confidently say that I could ditch all my other blacklists for that one, and be confident of seeing an end of 99% of the world’s spam.

[Author]

Posts