Amavisd/ClamAV/SpamAssassin with OS X Server 10.3.3
This article covers using amavisd to provide spam and virus controls to Postfix. This method will provide server-wide spam and virus filters for all incoming and outgoing mail. It’s a very robust system and can be easily configured to provide a number of different site-specific options for your users. However, combating unwanted e-mail is an ongoing war and not just a single engagement. This article will describe a good beginning to a full-featured mail system, but you should not stop here. Please take a look at David’s article on using RBLs and regular expressions to block unwanted mail with Postifx. That article will give you a number of other options that can be used in conjunction with what I’ll go through here for a very robust and flexible mail system.
This process works as a content filter for postfix. This means that Postfix will receive the e-mail and then pass it off, using an SMTP connection to port 1024 on the local machine, to amavisd before doing any actual processing of it. Amavisd will then run the mail through ClamAV, for virus detection, and then SpamAssassin, for spam detection. If the e-mail passes both sets of tests amavisd will then return the e-mail to Postfix by using an SMTP connection to port 1025 on the local machine. At this point Postfix will then deliver the mail to the POP/IMAP server that you are using.
This method was last tested using the stock Postfix/Cyrus mail system on OS X Server 10.3.3.
1 – Get Perl
If you are using OS X 10.2 you’ll need to first upgrade you installation of Perl to 5.8. You can grab this from Aaron Faby’s site. Download the perl updater and double-click to install. It’s important to note, since this is an article aimed at 10.3, that Perl 5.8 is already installed in 10.3 so you won’t need to do this.
2 – Get the AV system going
Now that you have Perl installed we can concentrate on getting the anti-virus part up and running. We’re going to use a ClamAV which is a free virus scanner that runs well on OSX. However, you are free to use one of a number of command line virus scanners that the Aamavis system supports.
Before we can get into downloading and compiling ClamAV we need to create a user for clamav, the av scanner, to run as. We’ll do this from the command line using niutil. Although you are more then welcome to do this through Workgroup Manager, or even the accounts pane in System Preferences. Just remember to make the password “*” so that no one can log into the system as this user.
We’ll be using user ID 37 for the ClamAV user. This should work for both 10.2 and 10.3. However, you can check and make sure this doesn’t conflict with any of your current users by using the “id” command. I’ll use the dscl command here because it’s something you should get familiar with if you are not already. If you are using an 10.2 machine you’ll need to substitute the appropriate niutil commands.
sudo dscl localhost -create /NetInfo/root/Users/clamav
sudo dscl localhost -create /NetInfo/root/Users/clamav uid 37
sudo dscl localhost -create /NetInfo/root/Users/clamav gid 37
sudo dscl localhost -create /NetInfo/root/Users/clamav home /tmp
sudo dscl localhost -create /NetInfo/root/Users/clamav passwd “*”
Now to finish up and add the group that we put the clamav user into.
sudo dscl localhost -create /NetInfo/root/Groups/clamav
sudo dscl localhost -create /NetInfo/root/Groups/clamav gid 37
Now you can download ClamAV and compile it. Version .70 was the latest at the time this article was written, but please check the site to see if it has been updated recently. Since the application is now hosted on SourceForge you’ll have to find your favorite mirror and download it from there. So point your browser here. Then unpack it.
cd clamav-0.7
./configure
make
sudo make install
cd ..
In previous versions of this setup I had you link the binaries to somewhere that was inside your path. This time around, since it’s highly unlikely that you will ever be calling these binaries by hand, we’ll leave them were they were installed and just use the full path when calling them in scripts.
Now for the moment of truth. Run the clamscan command against the folder that contains the ClamAV source. There’s a virus or five in there and if you’ve done everything correctly you will see them come up in the scan.
/usr/local/bin/clamscan -r -l scan.txt clamav-0.7
Finally we can set up the log files that clamav needs and start up freshclam which will download the latest virus files from the developer’s site six times a day. Later on we’ll be adding this to a startup item, so that it happens automatically.
sudo touch /var/log/clam-update.log
sudo chmod 644 /var/log/clam-update.log
sudo chown clamav /var/log/clam-update.log
sudo freshclam -d -c 6 -l /var/log/clam-update.log
sudo mkdir /var/run/clamav
sudo chown clamav /var/run/clamav
There are two ways to use ClamAV with our mail setup. The first is to allow amavisd to invoke clamscan on demand. This is the way to go if you plan on having a very low volume mail server. If you are getting more than a few e-mails an hour, and who isn’t these days, then you’ll be a little more efficient if you use the daemonized version of ClamAV, clamd, which will keep it running in the background.
If you go with the “on-demand” method, then you can move on to step 3. Otherwise you’re going to need to edit /usr/local/etc/clamav.conf.
Start by commenting out or removing the 8th line from the top witch contains just “Example” or it clamd won’t run. Then change
#TemporaryDirectory /var/tmp
to
TemporaryDirectory /tmp
And then:
LocalSocket /tmp/clamd
to
LocalSocket /var/run/clamav/clamd
You can now kick clamd off with this command:
sudo su clamav -c /usr/local/sbin/clamd
3 – Amavisd
Now you are done with ClamAV so we can move on to the glue that gets the virus and spam utilities to play well with Postfix. Again we’ll grab the latest version as of the writing of this article. Amavis is set up to act as a mini smtp server. Postfix will deliver the e-mail to Amavis which will do the appropriate scans on it. Then Amavis will send the e-mail back to Postfix which will then finish the delivery to the local user.
curl -O http://www.ijs.si/software/amavisd/amavisd-new-20030616-p8.tar.gz
gnutar -xzvf amavisd-new-20030616-p8.tar.gz
cd amavisd-new-20030616
Amavisd needs it’s config file and binaries in the right places and some space to work on the e-mail. So, being the nice admins that we are we’ll do this.
sudo cp amavisd.conf /etc/
sudo chown root /etc/amavisd.conf
sudo chmod 644 /etc/amavisd.conf
sudo cp amavisd /usr/bin/
sudo chown root /usr/bin/amavisd
sudo chmod 755 /usr/bin/amavisd
sudo mkdir /var/amavis
sudo chown clamav:clamav /var/amavis
sudo chmod 750 /var/amavis
sudo mkdir /var/virusmails
sudo chown clamav:clamav /var/virusmails
sudo chmod 750 /var/virusmails
sudo touch /var/amavis/whitelist_sender
You now need to edit your amavisd config file. You can find it at /etc/amavisd.conf. This file contains a huge number of options that will pretty much determine your spam and virus policies for your server. You should familiarize yourself with this file so that you get the desired results from this system. It’s rather well commented so you shouldn’t need to mess with it too much. Although there are a few directives that you will need to change before you are able to launch amavisd.
In section I you’ll need to change
$mydomain to your main e-mail domain.
$daemon_user should be set to “clamav”
$daemon_group should be set to “clamav”
Section II and III you can probably leave alone.
Section IV will require you to make some decisions. This section determines what happens when an e-mail is determined to be a spam or virus e-mail. Here you can specify the notification templates for what your bounce messages say. More importantly you an determine what you’ll do with spam and virus e-mails.
The final destiny variables are what you are interested in here. By default amavisd will bounce all spam back to the sender. You may find that this clogs up your mail system attempting to be nice to spammers. If that’s the case you can set this to D_DISCARD which will effectively delete the mail in question.
You will also want to set your $virus_admin and $spam_admin settings where the respective notifications will be sent.
The quarantine settings allow you to specify where the spam and virus e-mails will be stored. If you are interested in keeping the e-mails you can direct them to an e-mail address or folder, otherwise you can set these to “undef” which will delete the mails. If you do keep them, make sure you clean the folder out on a regular basis or else you’ll fill your drive up.
Section V sets up white and black lists for amavis. Use these to add in any domains that you know are good or bad.
Section VI you can leave alone.
Section VII is where you specify when e-mail is tagged as spam. The sa_tag levels determine when to quarantine spam mails and when to kill them. Also in this section you’ll want to uncomment the clamd section here, which should look like this:
[‘Clam Antivirus-clamd’,
&ask_daemon, [“CONTSCAN {}n”, “/var/run/clamav/clamd”],
qr/bOK$/, qr/bFOUND$/,
qr/^.*?: (?!Infected Archive)(.*) FOUND$/ ],
4 – More fun with Perl
Now we need to get some perl modules installed. CPAN makes this easy, but we will have to force one or two of them to go. I haven’t come across any problems with this in testing, but do keep an eye on things. Also, when you are installing these perl modules you may run across dependencies that you don’t have installed yet. Please respond in the affirmative when it asks you if you want them installed too.
sudo perl -MCPAN -e shell
Now you are in the CPAN system. You will then type in the next three commands which will install the modules. Some of these modules may ask if you want to install the dependencies. Say “yes” to this.
install Archive::Tar Archive::Zip Compress::Zlib Convert::UUlib MIME::Base64 MIME::Parser Mail::Internet Net::SMTP Net::Server Digest::MD5 IO::Stringy Time::HiRes Unix::Syslog Digest::SHA1 Mail::SpamAssassin
force install Convert::TNEF
Finally exit out of CPAN.
quit
Now a quick test to see if this all works. We’re going to switch to the “clamav” user and then start up amavisd in the debug mode.
sudo su clamav
amavisd debug
If all works well you’ll see a lot of log information finished up with “Parent ready for children.” That lets you know that you are good to go. Leave this window open with amavisd running so we can do some testing with it in a bit.
5 – Touch up Postfix
Almost done, we just need to edit the Postfix files. If you don’t have Postfix up and running already, stop what you are doing and read through our articles on doing that. Otherwise you can continue and make changes to the two Postfix config files.
First you need to add this line to /etc/postfix/main.cf. It will tell Postfix to run amavisd as a content filter before delivery.
content_filter=smtp-amavis:[127.0.0.1]:10024
Now add this to /etc/postfix/master.cf.
smtp-amavis unix - - y - 2 smtp
-o smtp_data_done_timeout=1200
-o disable_dns_lookups=yes
-o smtp_send_xforward_command=yes<br>
127.0.0.1:10025 inet n - y - - smtpd
-o content_filter=
-o local_recipient_maps=
-o relay_recipient_maps=
-o smtpd_restriction_classes=
-o smtpd_client_restrictions=
-o smtpd_helo_restrictions=
-o smtpd_sender_restrictions=
-o smtpd_recipient_restrictions=permit_mynetworks,reject
-o mynetworks=127.0.0.0/8
-o strict_rfc821_envelopes=yes
-o smtpd_error_sleep_time=0
-o smtpd_soft_error_limit=1001
-o smtpd_hard_error_limit=1000
-o receive_override_options=no_header_body_checks<br>
I’ve found that if you add these same changes to /etc/postfix/master.cf.default and to /etc/postfix/main.cf.serverdefault that the changes will not be overwritten when you use Server Admin.
6 – Now just to start everything up.
First fire off Postfix.
sudo postfix start
Then if you don’t still have the amavisd window up and running you can run amavisd in the background.
su clamav -c amavisd
Now put some mail through postfix. Perhaps some e-mails from support@microsoft.com that you may have received recently. Or maybe some offers from kind souls in Africa that found you through the Internet and think you can help them with their million dollar problems.
If you see the spam and viruses being stopped, things are going well. Take a look again at the amavisd config file and make sure the settings are what you want.
If you want all of this to get started at boot you’ll need to set a startup item for ClamAV and Amavis. You can either add these two commands to an existing startup item or roll your own.
su clamav -c amavisd
freshclam -d -c 6 -l /var/log/clam-update.log
Here’s a sample Startup Item script:
#!/bin/sh
. /etc/rc.common
ConsoleMessage "Starting Amavis anti-virus/spam daemon."
if [ ! -d /var/run/clamav ] ; then
mkdir /var/run/clamav
fi
chown clamav /var/run/clamav
/usr/bin/su clamav -c /usr/local/sbin/clamd
/usr/bin/su clamav -c amavisd
/usr/local/bin/freshclam -d -c 6 -l /var/log/clam-update.log
7 – Learn from your mistakes
A very cool feature of Spam Assassin is it’s ability to learn from it’s mistakes. You can feed false positives and negatives to Spam Assassin which will incorporate the spam into it’s Bayesian filtering database.
The simplest way to do this is to create two shared IMAP mailboxes. You can use cyradm to create these boxes and make them available to all users. This way a user can drag the e-mail from their inbox to either the Ham or Spam shared box. Dragging in a mail application is better then forwarding as that will loose the headers.
This script will read in a mailbox, you’ll need to give it the file path to the mailbox, and then adds those mails as either ham (actual e-mails erroneously marked as spam) or spam. It will then delete the e-mails in the box and reconstruct the Cyrus database. This script will only work with the Cyrus POP/IMAP server. So if you are using another package, you’ll need to look for another script.
#!/bin/bash
# thanks to Holger Dunkel for basis of this script
# culled from the suse-slox-e@suse.com mailing list
#
# 5/04 - Added CLI switches Joel Rennich
#
# Some defaults
IMAP_DIR=/var/spool/imap/user/
HAM=NO
SPAM=NO
#
# Now for the proceedures
#
# check for root privs
check_root() {
if [ `whoami` != "root" ]
then
echo "you need to be root to do this"
exit 0;
fi
}
# run sa-learn on spammed mailbox
learn_spam (){
dir=${IMAP_DIR}${MAILBOX}
c=0
if [ -d $dir ];then
cd $dir && echo now in $dir
for i in *. ; do
if [ -f $i ]; then
cat $i | su - clamav -c "sa-learn --showdots --spam --no-rebuild ";
echo $i, $c
# this removes emails from the shared folder
rm -v $i
let c++
fi
done
echo "processed $c mails"
else
echo $dir not found
exit 1
fi
}
# run sa-learn on not-spammed mailbox
learn_ham (){
dir=${IMAP_DIR}${MAILBOX}
c=0
if [ -d $dir ];then
cd $dir && echo now in $dir
for i in *. ; do
if [ -f $i ]; then
cat $i | su - clamav -c "sa-learn --showdots --ham --no-rebuild ";
echo $i, $c
# this removes emails from the shared folder
rm -v $i
let c++
fi
done
echo "processed $c mails"
else
echo $dir not found
exit 1
fi
}
#
# integrate learned stuff and rebuild mailbox
#
integrate (){
sudo -u cyrus /usr/bin/cyrus/bin/reconstruct
su - clamav -c "sa-learn --rebuild"
}
while getopts m:hs SWITCH
do
case $SWITCH in
h) HAM=YES;;
s) SPAM=YES;;
m) MAILBOX=$OPTARG;;
*) echo "sa-learn helper script to bring in Cyrus mail boxes"
echo "learn them as spam or ham and then delete the e-mails"
echo "and rebuild the box"
echo " "
echo "useage: $PROGRAM [ -h | -s ] [ -m mailbox directory]"
echo " "
echo examples:
echo "learn Joel's spam mailbox as spam"
echo " autosa -s joel/spam"
echo "learn Arek's ham mailbox as ham"
echo " autosa -h arek/ham"
esac
done
#
# Learn ham
#
if [ $HAM = "YES" ]
then
check_root
learn_ham
integrate
fi
#
# Learn spam
#
if [ $SPAM = "YES" ]
then
check_root
learn_spam
integrate
fi
<br>
For example
/usr/local/bin/autosa -h -m spamadmin/Ham
