Analyzing Amavis logs - UPDATED

Monday, September 27 2004 @ 12:50 pm MDT

Contributed by: rmleonard

Turn your spam into colorful art!

Use amavis-stats and rrdtool to chart out your spam/ham/virus history from your mail logs.

Firstly - this is a follow on to this article on Updated Spam/Virus controls with OS X Server. If you can't get that stuff working - this won't either... start there and let it churn for a few days then try this.

1. Update a few perl packages.

CPAN makes it easy.

sudo perl -MCPAN -e shell

Then from the cpan prompt

install Getopt::Std Time::localtime Time::Local

To see if other components are out of date enter "r" at the cpan prompt and most of the perl components will update without a problem.

If you want to try updating the whole shebang - try:
sudo perl -MCPAN -e 'CPAN::Shell->install(CPAN::Shell->r)'

2. Now install rrdtools which will turn your spam into pretty art.
curl -O http://people.ee.ethz.ch/~oetiker/webtools/rrdtool/pub/rrdtool -1.0.49.tar.gz tar -zxvf rrdtool-1.0.49.tar.gz cd rrdtool-1.0.49 ./configure --enable-shared make sudo make install sudo make site-perl-install

then because we want rrdtool to be available to "minimal users"
sudo ln -s /usr/local/rrdtool-1.0.49/bin/rrdtool /usr/bin/rrdtool

3. Now for Amavis-stats which will generate the data from your mail files that rrdtools will use.

You should have built the clamav user and groups in the install of clamav so we will use them again here.

curl -O http://rekudos.net/download/amavis-stats.tar.gz tar -zxvf amavis-stats.tar.gz cd amavis-stats-0.1.12 ./configure --with-amavis-user=clamav --with-amavis-group=clamav make sudo make install

Now you'll need to make some edits to the config file for amavis-stats.

sudo pico /etc/amavis-stats/apache.conf

Change the amavis-stats/apache.conf file to look like this:

Alias /amavis-stats /usr/local/share/amavis-stats/


    Options +FollowSymLinks
    AllowOverride None
    order allow,deny
    allow from all

Alternatively you could just include this in the mail httpd.conf file, or copy the apache.conf file to /etc/httpd/sites.

If you stick with /etc/amavis-stats/apache.conf make sure to include it in the main httpd.conf file by adding this line to httpd.conf:

Include /etc/amavis-stats/apache.conf

Now again make this available to minimal users:

sudo ln -s /usr/local/sbin/amavis-stats /usr/sbin/amavis-stats

The amavis-stats program needs a lockfile area - the program defaults to /var/lock so,

sudo mkdir /var/lock sudo chmod ugo+w /var/lock

Now to restart apache.

sudo apachectl graceful

4. Now to test things.

sudo su clamav amavis-stats -r -d /var/log/mail.log

This should get you lotsa stuff running across the screen. The -r and -d options are NOT necessary for building the cron entry later.

You should now have files in the /var/lib/amavis-stats directory similar to this:

ls -laF

total 120
drwxr-xr-x  7 clamav  clamav    238 22 Sep 08:42 ./
drwxr-xr-x  4 root    wheel     136 22 Sep 08:37 ../
-rw-r--r--  1 clamav  clamav  43036 22 Sep 08:42 1.rrd
-rw-r--r--  1 clamav  clamav      6 22 Sep 08:42 amavis-stats.count
-rw-r--r--  1 clamav  clamav      9 22 Sep 08:42 amavis-stats.names
-rw-r--r--  1 clamav  clamav     24 22 Sep 08:42 amavis-stats.seen
-rw-r--r--  1 clamav  clamav     65 22 Sep 08:42 amavis-stats.state

If the rrd file isn't there one of the above steps wasn't done right.

For the real test you should now be able to point a web browser to your box as such:

http://your-servername.here/amavis-stats/index.php

5. Finally bring it all home by automating this and putting it into your cron tab.

sudo pico /etc/crontab

add this line

55 * * * * clamav /usr/sbin/amavis-stats /var/log/mail.log

Which means that every hour - at 55 past - clamav will run the stats program.
br> UPDATE********************* you'll need to make sure that amavis-new has the appropriate output logging in order for amavis-stats to work correctly.

Here is a sample taken from my amavisd.conf file

$log_level = 2; # verbosity 0..5
$DO_SYSLOG = 0; # log via syslogd (preferred)
$SYSLOG_LEVEL = 'mail.debug';
$LOGFILE="$MYHOME/var/amavis.log";

# log template compatible with amavisd-new-20030616-p10:
$log_recip_templ = undef;
$log_templ = '[? %#V |[? %#F |[?%#D|Not-Delivered|Passed]|BANNED name/type (%F)]|INFECTED (%V)], #
-> [|,][? %i ||, quarantine %i], Message-ID: %m, Hits: %c';

the HTML may munge the above code - look in the amavisd-new build directory at the amavisd.conf-sample.

In my above example - I have amavisd now routing its output to a separate file and not to the mail log.
So the crontab needs to be altered to point to this new place.

55 * * * * clamav /usr/sbin/amavis-stats /var/amavis/var/amavis.log

Comments (9)

AFP548
http://www.afp548.com/article.php?story=20040922115018428