Is Bayesian filtering turned on?

[thinbits]

I’ve been dutifully training the SA filters by feeding it batches of spam and ham (in mbox format, 200+ at a time), however I don’t think it’s actually filtering anything.

SA is catching a lot of spam, and the X-Spam-Status header is being added with all the tests that ran. However, none of the 700+ spams I just looked at (caught and uncaught) have BAYES_XX in the X-Spam-Status header.

My /private/etc/mail/spamassassin/local.cf file has auto_learn 1 and use_bayes 1, among other settings.

Shouldn’t I be seeing some indication of the bayes filters in the mail headers?

Thanks,
Dave

[thinbits]

I still have not figured out why I’m not getting any bayesian filtering. I tried a test run of SA from the command line on a batch of known spam with the -D debug switch on.

su – clamav -c “spamassassin -D –test –local –mbox /Users/dave/spam3.mbox > spam.test”

debug: bayes: 2187 tie-ing to DB file R/O /var/clamav/.spamassassin/bayes_toks
debug: bayes: 2187 tie-ing to DB file R/O /var/clamav/.spamassassin/bayes_seen
debug: bayes corpus size: nspam = 4310, nham = 3017
debug: tests= BAYES_99,FORGED_RCVD_HELO,HTML_80_90,HTML_MESSAGE,, [lots of other stuff]…

So, when run manually, I can see that bayesian filtering works. It just doesn’t seem to be engaged when mail is normally received by postfix. I have yet to see a BAYES_XX tag added to any spam processed by postfix/amavis.

Any thoughts? Is there some way I can enable SA debug output when called from amavisd?

Thanks,
Dave

[Anonymous]

ping ping ping.

bump bump bump.

I just noticed the exact same thing. Why are we supposed to train our anti-spam software when it has no effect?

[thinbits]

I’ve added the “debug” arg to the launchd plist for amavisd, and turned on the sa_debug thing at the bottom of the amavisd.conf file. I now get a lot more logging from amavisd, but I still don’t see any debug logging from SA anyplace I’ve looked.

I’m suspecting there is some sort of problem with SA finding the correct bayes databases. Possibly it’s being run as a user other than clamav, or some sort of path or ownership problem. It would probably be obvious if I could figure out how to get SA to output debug logs when called from amavisd.

Dave

[Anonymous]

/private/var/amavis/.spamassassin/bayes_journal
/private/var/amavis/.spamassassin/bayes_seen
/private/var/amavis/.spamassassin/bayes_toks

Thar she blows, thinbits.

Those are the amavis users’ bayes files for SA.

On my system the file permissions look kosher. All owned by clamav:clamav and rw for it.

[Anonymous]

We’ve having the same exact problem. No Bayes on 10.4.2 server! Very strange. Has anyone been able to fix this?

[thinbits]

That’s interesting… I did a locate on bayes_seen and got the following:

/private/var/amavis/.spamassassin/bayes_seen
/private/var/clamav/.spamassassin/bayes_seen

Looking in those directories I see:

dave% sudo ls -l /private/var/amavis/.spamassassin/
-rw——- 1 clamav clamav 671744 Oct 14 08:13 auto-whitelist
-rw——- 1 clamav clamav 172032 Oct 14 08:03 bayes_seen
-rw——- 1 clamav clamav 1318912 Oct 14 08:03 bayes_toks

dave% sudo ls -l /private/var/clamav/.spamassassin/
-rw——- 1 clamav clamav 655360 Oct 14 01:00 bayes_seen
-rw——- 1 clamav clamav 5349376 Oct 14 01:00 bayes_toks
-rw-r–r– 1 clamav clamav 1175 Oct 9 18:32 user_prefs

From what I can tell, /private/var/clamav is where the results of learn_junk_mail go, (or at least that’s what happens on my machine). I’m wondering if it actually pulling from the /private/var/amavis directory when processing real mail. That would explain the lack of filtering…

I’ll tinker with this today and report my results. I’m hoping we are on to something…

Dave

[thinbits]

I stopped mail, moved /var/amavis/.spamassassin aside and replaced it with a symlink to /var/clamav/.spamassassin.

The next two mails that came in got a BAYES_99 tag. Yay!

So, it appears there is some sort of disconnect between the database path learning takes and what the mail server uses. I’ll let this run for the weeked and see how it turns out.

[Anonymous]

This is a huge bug. One of the features they advertise doesn’t even work! Do you think a symlink would work? Let me know how this is working for you.

[Anonymous]

This is a huge bug.

Way to go, Thinbits. You earned yourself a high-five. Someone want to blog this somewhere? I hope its fixed in 10.4.3

[thinbits]

Since placing the symlink, I’m getting almost 100% accuracy with spam filtering. It’s quite amazing! I’m not used to my Inbox not being full every morning.

I reported this as bug # 4305231 at bugreport.apple.com

Maybe if enough people report the bug it will get fixed for 10.4.3

[Anonymous]

Hmm. I’ve “moved /var/amavis/.spamassassin aside and replaced it with a symlink to /var/clamav/.spamassassin” but no effect. In fact, on my 10.4.2 machine, it seems like there is alot more info in amvis than clamav. Any suggestions?

[pblolo]

[QUOTE BY= thinbits]So, it appears there is some sort of disconnect between the database path learning takes and what the mail server uses. I’ll let this run for the weeked and see how it turns out.[/QUOTE]

Strange, I checked my server, I have only the good .spamassassin folder, which explains why it works fine here also.

Did you install in a special way ?

Here : 10.4.2 server, clean install.

[aderium]

I only have the .spamassassin in /va/amavis when i move it os .spamassassin_old it recreates it but none in /var/clamav

Also I noticed in /etc a /etc/spam with some files like amavisd.conf what is that there for ?

Thaks

P.S. talking about bugs anybody able to filter the logsi in serveradmin mail portion ?

[Anonymous]

[QUOTE BY= Aderium] I only have the .spamassassin in /va/amavis when i move it os .spamassassin_old it recreates it but none in /var/clamav

Also I noticed in /etc a /etc/spam with some files like amavisd.conf what is that there for ?

Thaks

P.S. talking about bugs anybody able to filter the logsi in serveradmin mail portion ?
[/QUOTE]

I was just about to submit that to apple. No, I cannot either. Filters work everywhere else that I have tried but not in mail.

[Author]

Posts