AFP548.com’s Anti-spam Measures

—by David O’Donnell, atropos@afp548.com

Part II in a series of articles. One of the most important things you can do as a system administrator is take steps to prevent spam (unsolicited commercial and/or bulk e-mail, USENET posts and the like) from polluting your server. This article, which is a work in progress, details the steps taken on AFP548.com’s server to prevent spam.

NOTE: The information in the files presented or accessible from this article is live—it represents the actual files on the AFP548.com server. Some material (particularly comments about specific spammers) may be offensive to sensitive people.

AFP548.com uses Postfix 1.1.1 as its mail transport agent, due to Postfix' flexibility and power. Postfix is inherently secure against basic exploitations used by spammers. In particular, Postfix comes armored against promiscuous third party relaying, the practice of stealing the resources of another party’s e-mail server in order to send mail to unsuspecting victims. Relaying most often comes about as the result of a site using an old mail transport agent, designed before spammers began thieving resources. In the early days of the Internet, mail servers permitted relaying as a means of ensuring mail got to its destination when network paths were neither as copious nor as strong as they are today. Today, of course, that is no longer the case. Unfortunately, a great many sites still run mail transport agents that are wide open for relaying, which forces other sites to put piecemeal blocks in (see below).

Postfix comes with a number of anti-spam controls. The figure below shows the segment of AFP548.com’s Postfix configuration file that deals with anti-spam controls:

Figure 1: AFP548.com’s Postfix anti-spam controls in main.cf

The display in Figure 1 is 'live'; it represents what is actually in the server’s configuration file, massaged somewhat courtesy of sed.

Lines beginning with octothorpes ("#") are comments and are ignored by Postfix' processor. Postfix is very flexible in how you can format its command lines, so it is easy to format the file to be readable by humans.

The first part of the anti-spam controls establishes the list of DNS-based 'realtime black-hole lists' (DNSBLs). The first and best known of these DNSBLs was the MAPS RBL, which is why the Postfix configuration directive is 'maps_rbl_domains'. Each host listed represents an individual DNSBL; we do not use the actual MAPS RBL any longer on account of their overly complex licensing requirements. If you meet their requirements, however, you can include their DNSBL by adding 'blackholes.mail-abuse.org' to the list shown in Figure 1.

The line 'smtpd_helo_required = yes' requires all incoming mail transactions to begin with the SMTP command HELO hostname (preferably, the ESMTP command EHLO). This requirement will block some spam software, written by authors who do not bother to read or follow IETF standards or RFCs.

The line 'smtpd_delay_reject = no' makes Postfix' response to invalid conditions fast, rather than slow. Unfortunately, this may cause problems for badly-designed client software on the other end, but since they're likely spammers that’s just too bad.

Skip past 'smtpd_recipient_restrictions' for now; the line 'allow_untrusted_routing = no' helps to secure promiscuous relaying even further, by ensuring that no form of relaying will be permitted from an untrusted source, even if the destination is a domain that Postfix is otherwise authorized to use as a relay destination. While this is the default value, it’s not a bad idea to include it anyway.

The 'header_checks' line specifies one or more files that Postfix uses as content filter controls for post-receipt but pre-delivery checking. For our configuration, we are using a regular expression lookup file, rather than a database file. The contents of our 'spammerheaders' lookup table is shown in Figure 2:

Figure 2: AFP548.com’s 'spammerheaders' header-filter table

The question that may leap to mind is, “why are you checking for spam after you accept the SMTP transaction?” While it’s true that the more effective means of blocking the thieves is to refuse connections, it’s not always feasible or possible. Eventually, lists of IP addresses become unwieldy to manage. IP-only blocks become ineffective when the incoming e-mail is routed through another host, as well, and header_checks allows us to add in pre-delivery filters for those messages as well.

The format for regular expression matching is straightforward: pattern action response. As you can see, we currently search on From, Received and Subject headers, matching pernicious spammers who rotate among numerous providers and IP addresses. Personally, I think our response text is rather more pleasant than they deserve!

The real meat of our restrictions comes in the 'smtpd_recipient_restrictions' line, as shown in Figure 3:

Figure 3: AFP548.com’s main-gun restrictions line from main.cf

Since the number of people who use this server for outgoing mail is quite small, and all of them are trusted parties, we’re only placing restrictions on incoming e-mail. If your server has many users, some of whom may succumb to the Dark Side of spamming, you should look into rate limiting on your outgoing connections.

Postfix allows even more granular restrictions than just recipient-based, but for AFP548.com that proves problematic as several valid foreign originators are unfortunate enough to be forced to use external mail servers that are blocked by our DNSBLs. By limiting granularity in this fashion, we can punch specific access holes for them while continuing to thwart many spammers' desires. You should check out this page for a more extensive list of all of Postfix' anti-spam controls.

Explanations of our various restrictions follows; if the table lacks one or more entries or has one or more extra compared to Figure 3, it’s because I haven’t had a chance to update this article. Figure 3 is the exact current list of restrictions we use (though not necessarily in the order they appear in the file):

reject_unknown_sender_domain

Postfix checks the sender-supplied mail address (in the MAIL FROM portion of the SMTP transaction) for valid A or MX DNS records. If none are found, the mail is rejected.

reject_unknown_recipient_domain

Postfix will reject mail if the supplied recipient domain lacks a valid A or MX DNS record.

permit_mynetworks

Postfix will accept the e-mail transaction if the destination matches one of the entries in the $mynetworks variable in main.cf. This is particularly useful when your server is acting as a central mail router for machines within your network space.

reject_unauth_destination

Postfix will reject the e-mail unless the mail is directed toward a host that Postfix considers valid, such as one for which your server relays, or a virtual domain that Postfix manages.

check_sender_access hash:/etc/postfix/access-addresses check_sender_access hash:/etc/postfix/access-ips check_client_access hash:/etc/postfix/access-addresses check_client_access hash:/etc/postfix/access-ips

Postfix checks two databases against the supplied sender and recipient addresses. In this case, the databases are hash tables that match against addresses or IP addresses. These databases are covered in greater detail below.

reject_unauth_pipelining

Postfix will reject mail sent by poorly-written SMTP software that doesn’t understand how to perform command pipelining. Generally speaking, spammerware meets the description of 'poorly-written SMTP software'

reject_invalid_hostname

Postfix will reject the e-mail if the hostname supplied in the HELO or EHLO command in the SMTP transaction is formatted improperly, a common spammer tactic.

reject_non_fqdn_hostname

Postfix will reject the e-mail if the hostname supplied in the HELO or EHLO command is not a fully-qualified domain name, another common spammer tactic.

reject_maps_rbl

Postfix polls any hosts listed in the maps_rbl_domains definition to see if the IP address of the server attempting to deliver mail to us is listed there. If so, the mail is rejected.

reject_unknown_client

Postfix will reject the e-mail if the machine attempting delivery doesn’t have a PTR DNS record. (Basically, this will block connections from random IP addresses without any associated name information)

permit

The terminal condition—any mail that isn’t rejected by any of the conditions above will be accepted.

If you think these rules make AFP548.com pretty mean on spammers, well—they generally do! The downside to this sort of restrictive behavior is that it becomes increasingly possible to catch legitimate e-mail that comes from machines that are poorly configured. For example, friends who are forced to mail servers that are legitimately listed as open relays; or users new to the whole Internet server game who haven’t quite figured out that IP addresses should have some sort of name entry associated with them.

Luckily, there are ways around this, although in general it’s best to try to get senders to fix things on their end anyway. Postfix processes the restrictions in linear order, and stops processing at the first match. Thus it’s important that you put explicit exceptions to the rules in the files used for check_sender_access and check_client_access (and that those checks appear high up in your list of restrictions).

Two files are used as the bulk of our countermeasures against spam: access-addresses and access-ips. These files, which grew out of a master 'access' file back in the days when the mail currently handled on AFP548.com’s server was being managed by Sendmail 8.x on a RedHat Linux box, specify the hostnames and IP addresses that we explicitly block (and in some cases permit). The master access file got so large and unwieldy that it eventually got split in two. Figure 4 shows the first forty lines of access-addresses; Figure 5 shows the first forty lines of access-ips.

Figure 4: The top part of AFP548.com’s access-addresses anti-spam filter

Figure 5: The top part of AFP548.com’s access-ips anti-spam filter

To see the full files, click either of these links: access-addresses and access-ips.

The format of entries in each file is quite simple: first there is the pattern against which matching will be made, then some whitespace, and the desired action when a match is met. Lines where the action is “OK” or “PERMIT” represent specific holes in our anti-spam armor, and as such they are few and far-between. The more interesting lines are the rejection notices—of which there are many.

The access-addresses file contains matches on e-mail addresses and domains. These are effective against mail deliveries that are direct to our mail server (otherwise, their patterns would be included in spammerheaders, above). To reject mail, Postfix lets you specify either the action REJECT, or an SMTP error code followed by explanatory text. Since AFP548.com shares its server with several other domains, rejection notices point the recipient at a web page (http://www.fates.org/SMTP550.html) with an explanation of why their mail is being rejected. Lines beginning with the octothorpe ("#") are comments, intended only for the server administrator, and as such the may be a bit piquant. The usual format is DDMMYYYY comment, though the oldest entries do not have any associated comments.

The access-ips file matches on IP addresses and subnets. Entries in this file fall into two categories: open relays and spam-originators. Open relays tend to be listed with their full associated e-mail address, in the hopes that the responsible system administrators will eventually plug the hole and contact us to have the block lifted. Spam-originators can be either IP addresses or subnets. Each IP address is made up of four octets; they read from right to left in decreasing degree of granularity. For example, the address, with four octets, 192.168.0.1 represents a single machine (for our purposes); the address 192.168.0 represents a subnet of 254 machines; the address 192.168 represents 64,516 machines, and so forth.

Blocking a single IP is usually only necessary when an individual amateur spammer has gotten hold of an IP address on her or his machine; or when an administrator is unreachable. Blocking larger segments of the Internet is only called-for when the organization is unwilling or unable to prevent its machines from being used for spamming.

NOTE: As your blocking list grows, it is important that you periodically clean it out, to limit the number of stale blocks.

There you have it—the full scope of AFP548.com’s anti-spam provisions!