Combatting Spam with Mac OS X Server

Part 1: Apple Mail Service

—David O’Donnell, atropos@afp548.com

NOTE: This article is obsolete for Mac OS X Server 10.3 and later.

What is “spam"? Definitions vary, but “spam” can generally be described as e-mail that is unsolicited, often sent in bulk and frequently commercial in nature. Chances are if you have been using the Internet for more than a few weeks, you’ve experienced the effect of spam first hand: unbidden, e-mail starts appearing in your inbox. Sometimes, it’s someone trying to hawk a product or service; sometimes, the messages try to sell spamming services, adult services, or worse. Spamming has been around for years. On the Internet, a pair of sleazy lawyers sent advertisements for a “service” claiming to improve the chances of immigrants to get Green Cards to the USA to virtually every USENET newsgroup in existence; this affront to news readers was the first large scale spamming incident. Spamming came to e-mail when underhanded individuals styling themselves as “entrepreneurs” discovered that—relative to the price of traditional marketing methods—sending e-mail was practically free. They began collecting e-mail addresses and sending out hundreds of thousands of pieces of unsolicited e-mail.

Spam has become such an intrusive, infuriating problem that state and national legislatures are enacting or pursuing laws specifically designed to protect individuals from being inundated with the stuff.

Spamming affects Mac OS X Server administrators in the following ways:

As a Mac OS X Server administrator using Apple Mail Service, here are some steps you can take to help limit the effect of spammer assaults on your server:

1. Secure Apple Mail Service from Relaying

Mac OS X Server ships with Apple Mail Service as its default Mail Transport Agent. As of OS X Server 10.1.2, this mail server is wide open for relay rape: the process of hijacking the services of an otherwise innocent server for the purpose of sending spam.

To secure Apple Mail Service against relay rape, launch Server Admin and log in. Click the “Internet” tab. Click and hold the Mail Service icon until the menu appears. You should see a layout similar to the image in Figure 1.

Configure Mail Server
Figure 1: Configure Mail Service

Click the “Filter” tab. Select the “Check incoming SMTP Connections” checkbox. This will cause the Apple Mail Server to perform checks against all connections to it. These checks are performed using one or more special “DNSBL” (Domain Name Server Blackhole List) DNS servers. If the incoming connection is matched to tables on the DNSBL server, the connection is rejected.

NOTE: in order to use the MAPS RBL to check incoming connections, you will need to obtain a contract from MAPS. Other DNS-based filter services exist and are covered in another article.

Close the Configure Mail Service window. Click on the Mail Service icon and select the “Configure Host Settings” menu item. Select the “Deliver mail to local addresses only (no SMTP relay)” and “Log recipient rejections to error log” checkboxes. Then click the “Save” button. Make sure to stop and restart Mail Service to ensure your changes are retained.

2. Force Mail Origination to Come from Valid Local Users

Open the Configure Mail Service window if you closed it earlier, and click on the Filter tab. Check the 'Require local “From” addresses to exist in Users & Groups' checkbox. This setting requires that mail origination requests must appear from valid local users, otherwise the mail will be rejected. Thus, if a spammer connects to your mail server and tries to relay off it by submitting a non-local (or local, but mail-disabled) user, the mail will be rejected.

NOTE: this is not a foolproof anti-relay measure. If you have enabled e-mail for local accounts (e.g., postmaster), the spammers can still relay their messages by using the valid local user as the 'from' address. Thankfully, Apple Mail Service won’t permit spoofed Received headers and will cheerfully display the proper originating IP address to the recipient—so long as the recipient is able to actually view Internet headers on their incoming mail.

3. Block Spammers from Sending to your Server

On the “Filter” tab of the Configure Mail Service window, make sure you select the “Log connection if SMTP name does not match IP address” and “Reject if name does not match address” checkboxes to perform an additional relay-based check on incoming mail connections. Spammers make use of mail servers that permit open relaying by connecting to the victim mail server and using it to act as a quiet forwarding service. By checking these two options, you require incoming mail to have an exact match between the DNS name of the connecting mail host and its IP address’ reverse-DNS entry. If there is no match, the connection is immediately closed.

Finally, you can use an extremely limited form of local blackhole by checking the “Reject messages from SMTP servers in list” checkbox, then clicking the Edit List? button. You can add named servers to that list (e.g., afp548.com) but not IP addresses. While this will help block some of the less intelligent spammers, it is not particularly useful for “chickenboners” and other spammers who make use of the SMTP protocol more stealthily.