Articles September 27, 2004 at 11:50 am

Analyzing Amavis logs – UPDATED

Turn your spam into colorful art!

Use amavis-stats and rrdtool to chart out your spam/ham/virus history from your mail logs.Firstly – this is a follow on to this article on Updated Spam/Virus controls with OS X Server. If you can’t get that stuff working – this won’t either… start there and let it churn for a few days then try this.

1. Update a few perl packages.

CPAN makes it easy.


sudo perl -MCPAN -e shell

Then from the cpan prompt


install Getopt::Std Time::localtime Time::Local

To see if other components are out of date enter “r” at the cpan prompt and most of the perl components will update without a problem.

If you want to try updating the whole shebang – try:

sudo perl -MCPAN -e 'CPAN::Shell->install(CPAN::Shell->r)'

2. Now install rrdtools which will turn your spam into pretty art.

curl -O http://people.ee.ethz.ch/~oetiker/webtools/rrdtool/pub/rrdtool
-1.0.49.tar.gz
tar -zxvf rrdtool-1.0.49.tar.gz
cd rrdtool-1.0.49
./configure --enable-shared
make
sudo make install
sudo make site-perl-install

then because we want rrdtool to be available to “minimal users”

sudo ln -s /usr/local/rrdtool-1.0.49/bin/rrdtool /usr/bin/rrdtool

3. Now for Amavis-stats which will generate the data from your mail files that rrdtools will use.

You should have built the clamav user and groups in the install of clamav so we will use them again here.


curl -O http://rekudos.net/download/amavis-stats.tar.gz
tar -zxvf amavis-stats.tar.gz
cd amavis-stats-0.1.12
./configure --with-amavis-user=clamav --with-amavis-group=clamav
make
sudo make install

Now you’ll need to make some edits to the config file for amavis-stats.


sudo pico /etc/amavis-stats/apache.conf

Change the amavis-stats/apache.conf file to look like this:

Alias /amavis-stats /usr/local/share/amavis-stats/


    Options +FollowSymLinks
    AllowOverride None
    order allow,deny
    allow from all

Alternatively you could just include this in the mail httpd.conf file, or copy the apache.conf file to /etc/httpd/sites.

If you stick with /etc/amavis-stats/apache.conf make sure to include it in the main httpd.conf file by adding this line to httpd.conf:


Include /etc/amavis-stats/apache.conf

Now again make this available to minimal users:


sudo ln -s /usr/local/sbin/amavis-stats /usr/sbin/amavis-stats

The amavis-stats program needs a lockfile area – the program defaults to
/var/lock so,


sudo mkdir /var/lock
sudo chmod ugo+w /var/lock

Now to restart apache.


sudo apachectl graceful

4. Now to test things.


sudo su clamav

amavis-stats -r -d /var/log/mail.log

This should get you lotsa stuff running across the screen. The -r and -d options are NOT necessary for building the cron entry later.

You should now have files in the /var/lib/amavis-stats directory similar to this:

ls -laF

total 120
drwxr-xr-x  7 clamav  clamav    238 22 Sep 08:42 ./
drwxr-xr-x  4 root    wheel     136 22 Sep 08:37 ../
-rw-r--r--  1 clamav  clamav  43036 22 Sep 08:42 1.rrd
-rw-r--r--  1 clamav  clamav      6 22 Sep 08:42 amavis-stats.count
-rw-r--r--  1 clamav  clamav      9 22 Sep 08:42 amavis-stats.names
-rw-r--r--  1 clamav  clamav     24 22 Sep 08:42 amavis-stats.seen
-rw-r--r--  1 clamav  clamav     65 22 Sep 08:42 amavis-stats.state

If the rrd file isn’t there one of the above steps wasn’t done right.

For the real test you should now be able to point a web browser to your box as such:


http://your-servername.here/amavis-stats/index.php

5. Finally bring it all home by automating this and putting it into your cron tab.


sudo pico /etc/crontab

add this line


55 * * * * clamav /usr/sbin/amavis-stats /var/log/mail.log

Which means that every hour – at 55 past – clamav will run the stats program.
br>

UPDATE*********************
you’ll need to make sure that amavis-new has the appropriate output logging in order for amavis-stats to work correctly.

Here is a sample taken from my amavisd.conf file

$log_level = 2; # verbosity 0..5

$DO_SYSLOG = 0; # log via syslogd (preferred)
$SYSLOG_LEVEL = ‘mail.debug’;
$LOGFILE=”$MYHOME/var/amavis.log”;

log template compatible with amavisd-new-20030616-p10:

$log_recip_templ = undef;
$log_templ = ‘[? %#V |[? %#F |[?%#D|Not-Delivered|Passed]|BANNED name/type (%F)]|INFECTED (%V)], #
-> [|,][? %i ||, quarantine %i], Message-ID: %m, Hits: %c’;

the HTML may munge the above code – look in the amavisd-new build directory at the amavisd.conf-sample.

In my above example – I have amavisd now routing its output to a separate file and not to the mail log.
So the crontab needs to be altered to point to this new place.


55 * * * * clamav /usr/sbin/amavis-stats /var/amavis/var/amavis.log

No Comments

  • I think you are missing some instructions in this

    curl -O http://rekudos.net/download/amavis-stats.tar.gz
    ./configure --with-amavis-user=clamav --with-amavis-group=clamav
    make
    sudo make install
    

    Should it read

    curl -O http://rekudos.net/download/amavis-stats.tar.gz
    tar -zxvf amavis-stats.tar.gz
    cd amavis-stats-0.1.12
    ./configure --with-amavis-user=clamav --with-amavis-group=clamav
    make
    sudo make install
    

    Sam D

  • Another question, it seems to be working for me now, but I do not see spam
    stats on the page. According to http://rekudos.net/amavis-stats/node/view/
    173 I need to make a change to the config file. Do you know what the
    change is?

    • I’m not sure, but I think you need to enable the PHP module for Apache. You can do this via ServerAdmin.

  • everything seems to be working, but i am getting this in system.log:
    Sep 30 22:55:00 server CRON[27103]: (clamav) CMD (/usr/sbin/amavis-stats
    /var/log/mail.log)
    Sep 30 22:55:05 server lmtpd[27114]: auxpropfunc error -1
    Sep 30 22:55:05 server lmtpd[27114]: unable to post for user: clamav, mail
    account is disabled
    Sep 30 22:55:05 server lmtpd[27114]: unable to post for user: clamav, mail
    account is disabled

    any ideas?

  • In step 4:

    sudo su clamav amavis-stats -r -d /var/log/mail.log

    Should the command actually be:

    sudo su clamav -c "amavis-stats -r -d /var/log/mail.log"?

  • looks like perl has been updated in the last few days, Getopt::Std no longer installed via cpan, instrad offering to update the entire perl installation. not sure if i want to do that as it might cause new problems..

    any one know if we can get the previous Getopt::Std and install it manually?

    I have amavis-stats running without it, generating the rrd files, but not the images for the web page.



    bryan

  • well, it turns out that i forgot to make the symbolic link to the rrdtool binary.

    the Getopt::Std problem was a wild goose chase.



    bryan

  • just a quick note that the development version of amavis-stats (amavis-stats-0.1.13-rc6) works nice on 10.3.

    🙂



    bryan

Leave a reply

You must be logged in to post a comment.