Spamfighting is very important for reducing the amount of spam we'll all receive in the future but it doesn't do much to affect your spam intake for today. This section looks at some popular methods that are used to reduce the amount of spam currently ending up in mailboxes.
7.1 How can an individual reduce the amount of spam they get?
7.1.1 How do spammers get our email addresses?
The obvious way to reduce the amount of spam you receive is to make sure that spammers don't have your email address! Before we can go further with this, however, we must learn how spammers get hold of email addresses in the first place. As it turns out, there are five main ways:
They pick them up when they're used publicly on the Internet, e.g. in a newsgroup posting or on a webpage. This is by far the most common way, and is known as "harvesting". Using your email address in a newsgroup or on a webpage is generally understood to solicit personal, topical replies from individuals, but is not a solicitation to receive broadcast advertising.
They buy a CD of addresses from another spammer. These addresses were probably harvested from newsgroups or webpages in the manner described above, and are often years out-of-date to boot. As the saying goes, there is no honour among thieves...
They guess them. For example, it's a fair bet that "firstname.lastname@example.org" could be a valid email address, although there's no way of knowing to whom it leads. When spammers concentrate this technique on one domain it is sometimes called a "dictionary attack". (As it happens, email@example.com isn't a valid email address, because "example.com" is a domain reserved for testing and examples.)
Our ISPs sell them our email addresses. This is extremely rare.
For a more detailed look at how spammers find email addresses, have a look at these documents:
7.1.2 Choose a non-obvious email address
Some spammers guess email addresses, so it may be a good idea to use something that spammers can't guess easily. For example, instead of firstname.lastname@example.org, why not have email@example.com?
7.1.3 Be careful with your email address
The only way to totally eliminate the chance of receiving spam is not to have an emailbox. Even if you have an emailbox and never ever show your email address to anyone else, there's still the chance that a spammer might guess your email address. However, there are a few less extreme steps you can take to at least reduce the amount of spam you receive...
Never, ever give your email address to a company you do not trust entirely. If in doubt, open a free email account with a web-based provider such as hotmail.com and use that address for communicating with the company; that way, if they do spam, you can close the account and you've only lost a free email account you weren't using for anything else.
Never, ever post to usenet using an unmunged email address you care about. Use a throw-away address from a free email provider or munge your email address as described in 7.1.4. (Some people have reported that you can reduce spam without impacting upon the ease of contacting you, by posting with a munged From: address or an unmunged Reply-To: address, but I can't believe the spammers won't catch on to this eventually.)
Never, ever allow your email address to appear on a website, including on a web-based discussion board.
Some people concerned about privacy enter made-up email addresses into online application forms and the like. This seems like a good idea, but it is important to make sure that the made-up domain you use doesn't actually belong to anyone, otherwise you'll just be sending spam to the innocent third-party who owns it. This can become a very serious problem for the owners of some domains popularly used in such forms.
BAD MADE-UP EMAIL ADDRESSES
GOOD MADE-UP EMAIL ADDRESSES
There are several free mail-forwarding services that can be used to reduce your spam-level. The idea is simple; you give a different mail forwarding email address to each company that asks for your email address, and the mail forwarder forwards all mail to these addresses to your usual mailbox. If a company ever starts to spam you, you just disable the forwarding address you gave them and you won't get their spam, without affecting your other incoming mail. Companies who provide this service include:
7.1.4 Address Munging
"Munging" is the act of mangling your email address so that it can still be read by a human but cannot be automatically harvested by spammers.
For example, my email address:
Could be munged into any of the following:
When munging, you have to be careful not to accidentally munge your own email address so that it's identical to someone else's, and should always munge the bits to the RIGHT of the @-sign and not just the bits to the LEFT (otherwise your ISP will still get your spam even if you don't yourself). Also, you should ensure that your munged domain name is NOT an existing domain (else the poor sod who owns it could get your spam).
Recent drafts of the Usenet message format RFC specifies that the From: line of a newsgroup posting must contain either a valid email address or an email address ending in ".invalid". Your munged email address should really comply with this forthcoming standard, e.g.:
Note that some spammers now have harvesting software that can remove widely-used munges like "NOSPAM".
Some ISPs forbid their customers from using a munged email address. In these cases, whitelisting can be an alternative. In this, you set up your mail account such that some given word or string of characters must be in the subject line for any mail to be accepted, and then you explain this in any newsgroup postings and webpages containing your address. This way people can respond to you, but spam will be deleted from the server without you having to spend time downloading and reading it. This works especially well with webpages, e.g. use:
<A href="mailto:firstname.lastname@example.org?Subject=FRIENDLYMAIL: Comments about my webpage">
Send me email!</A>
Then kill any mail that doesn't have FRIENDLYMAIL: in the subject line and have the rest forwarded to your real email address.
There have always been people who have filtered spam using simple rules in their email client; for example, depending on your tastes, it may be a fair bet that any message with "FREE LIVE SEX" in the subject-line is spam, and can be deleted or filtered into a separate folder that the user will clean out by hand. However, this has always been a somewhat hit-and-miss approach, requiring hard work and made more difficult by the somewhat crude filtering capabilities of many popular mail programs.
More recently, personal spam-filters have started to appear. These sit between your mail program and your mailbox, using more advanced methods to filter or tag likely spam messages. The number of personal spamfilters has skyrocketed in recent months; I even wrote one myself (SpamPal). Most of them work in different ways, and will have differing strengths and weaknesses. Here's a few links to get you started:
Free spam-filters for Windows users:
Commercial/Shareware spam-filters for Windows users:
Spam-filters for Unix users:
Spam-filters for Macs users:
There are also various companies who will filter the spam from your mail without the use of additional software. These include:
22.214.171.124 What is Bayesian filtering?
Bayesian Probability Filtering is an increasingly popular spam-filtering technique which has been integrated into popular email programs such as Mozilla. The idea is that you "train" the filter to recognise spam from non-spam, by telling it whenever it makes a mistake. This can be quite succesful because everyone's spam is different and the types of legitimate mail everyone gets is different; for example, anything I get that mentions "Viagra" may be spam, but another person may have a bedroom issue and legitimately need to discuss Viagra with someone. (Or vice versa.) The down-side to Bayesian filters is that it takes an appreciable effort to train the them; pre-trained Bayesian filters aren't really practical.
126.96.36.199 Challenge-Response Tools
Challenge-response systems, also known as "Reverse Whitelisting" or "Permission-based" fitering, take a different approach to traditional spam-filters. Whereas traditional filters start from a stand-point that all mail is good then try to detect the spam, Challenge-Response systems start by assuming all mail is spam then only letting through people on a "whitelist". If the user receives mail from someone not on a whitelist, the system "holds up" the mail and sends a "challenge" message to the sender. If sender replies ("responds") to the "challenge" message, the original message is "released" and allowed into the user's mailbox, and the sender is "whitelisted" so any future emails will be allowed through without this rigmarole. The theory here is that the spammers won't bother to reply to the "challenge" - most of them are using forged email addresses so they won't even receive the "challenge".
Put like that, it sounds like quite a good idea. But the simplicity of the solution doesn't reflect the complexity of the real world, and challenge-response has a number of problems:
Mailing lists, especially discussion lists. If I send a message to a mailing list with 1000 subscribers, would I receive - and have to respond to - 1000 challenge messages? Many Challenge-Response systems allow the user to whitelist a mailing list automatically, but this can be unreliable (and judging by experience plenty of people forget).
Automated mailings - generated by a computer with no human intervention - have no human sender who can respond to the challenge message. This immediately breaks things like password reminder messages, confirmed opt-in mailing lists, Cron job notifications and so forth. Again, these things could be whitelisted manually - but you have to remember, and anyway guessing the email addresses most of them will be sent from would be difficult.
Forged sender addresses. Spammers often forge the addresses of enemies or just random individuals as the senders of their spam - if a spammer forges me as the sender of a 1,000,000-recipient spam-run, the last thing I want to receive is a "challenge" message from each and every victim!
And of course, simple challenge-response systems can be fooled if the spammer stops using forged email addresses and sets up a simple bot to reply to the challenges. It has been suggested that challenge-messages could include a graphic image containing a number that has to be typed into the subject of the response, in order to prevent automatic responding, but this breaks the system for blind users and adds an extra hoop for senders to jump through. While it's tolerable if you only communicate with one or two new people every day, if you're (like me) exchanging emails with many new people every day (if you work in support, for example) then going through a prolonged challenge-response procedure for everyone - or even a fair proportion of senders - would be an enormous pain at best.
7.1.7 If I use a tool to send "bounce messages" for any spam I get, will I get less spam in the future?
When you send an email message to an address that doesn't exist, you receive a "bounce message" back. (If you've never seen a bounce message, try sending an email to "email@example.com" and you'll get one back within minutes.) There's a school of thought that says that if you could somehow send fake "bounce messages" in response to the spam you receive, spammers will remove you from their mailing lists and you'll get less spam in the future. To this end, there are various tools - the most well-known being MailWasher - that will generate such "fake" bounce messages.
The general consensus on news.admin.net-abuse.email is that this is a bad idea. Here's a few reasons why:
There is lots of anecdotal evidence that suggests spammers as a rule are not interested in removing dead email addresses from their lists - for example, The Story of Nadine.
The return address in almost all spam messages these days is forged, probably because the spammer knows his mailing list has lots of bad addresses and he doesn't want the bounce messages to fill up his own mailbox. So any "fake bounce" you generate probably won't reach the spammer anyway.
So at best, your "fake bounce" will hop around between mailservers consuming computing resources before being quietly dropped. However, a lot of spammers forge their spam to look like it came from the email address of a real person - either someone who's annoyed them (e.g. an anti-spammer) or just some poor soul picked at random. So your fake bounce message - together with those of everyone else who uses such a tool - would end up in the mailbox of this entirely innocent third-party. (My own email address has been forged in this way and let me tell you it isn't a pleasant experience - I have no idea how many of the thousands of bounce messages I received were real and how many were fake, but the last thing I'd have needed to receive were even more.)
By examination of the headers and included information in a bounce message, it's possible to make a reasonable inference as to whether it is real or fake. So even if your bounce message did somehow reach the spammer, his systems may well be able to figure out that it's fake and ignore it appropriately.
7.2 How can an ISP reduce the amount of spam their customers get?
7.2.1 Stop Accepting All Email
This will immediately reduce the spam intake of their customers to zero. Unfortunately, it also destroys email as a usable communication medium. In order to prevent this becoming necessary whilst still taking action to reduce their customers' spam levels, many ISPs adopt policies that are midway between blocking everything and doing nothing...
One tactic used by some ISPs to cut down on spam is filtering. The ISP scans incoming mail and any messages that match the pattern of a known piece of spam are discarded. The big danger with filtering is that of false positives; users are unlikely to be very pleased if some non-spam mails are mistaken for spam by the filter and never arrive.
Some of the filtering techniques discussed in 7.1.6 can also be applied across an entire I.S.P., although there may be additional risks due to questions of scale.
DCC (Distributed Checksum Clearinghouse) is based upon a very simple idea - if only we knew what email everyone was getting, we could detect what was bulk and what was personal. DCC works by collecting "checksums" of incoming messages (and not the email messages themselves) in distributed databases, and counting the frequency with which each checksum occurs. Using this information, spam can be filtered out. The down-side is that solicited bulk email must be whitelisted or it too will be filtered out.
The DCC code is currently available for a variety of Unix-like systems, and is intended to work best when installed close to the mail server.
7.2.3 DNSBL lists/Blackholing
Blackholing (or Blacklisting) is a variation on filtering whereby an ISP refuses to accept any email from machines that have a reputation for producing a disproportionate amount of spam. Many administrators have had some success with this tactic, although there are two main problems with it: firstly, someone will have to add more spam-sending machines to their list as more emerge if the effectiveness of the list is to be maintained, and secondly it is hard for the ISP to know when a machine on the list has reformed and is no longer emitting spam.
Of course, with any type of blackholing, any legitimate email from machines on the blackhole list will be lost along with the spam emails.
The main tool for blackholing are so-called DNSBL Lists. These are publically available lists of IP addresses that can be queried using a DNS lookup. There are a wide variety of DNSBL lists listing IP addresses according to various criteria; an individual site will have to choose the services to use based upon their own requirements. It isn't possible for me to discuss or link to every single DNSBL service, but I will cover a few that are most frequently discussed in the newsgroup.
But first, a word of warning. If you configure your server to use an external listing service you are turning over part of the control of your server to that service. You should exercise caution when you do this, and keep an eye on how the list is being used. If you have no means of your own to verify the integrity of the service you should pay some attention to a newsgroup such as news.admin.net-abuse.email and be alert for any reports that the service you have chosen has started to slip in quality.
Mail Abuse Prevention Systems LLC is a Californian company who were one of the pioneers of DNSBL lists. They offer a number of different services, including the famous RBL (Realtime Blackhole List), DUL (Dialup Users List), RSS (Relay Spam Stopper), and NML (Nonconfirmed Mailing List).
MAPS have fallen out of favour with many regulars of news.admin.net-abuse.email since they stopped making their services freely available. Users now require a static IP address, and need to sign a contract (although there is no monetary fee for individual and hobbiest sites). However, they are still used by many thousands of Internet sites, and have a reputation for causing a minimum of collateral damage.
relays.osirosoft.com initially earned its reputation for listing listing open relays (insecure mailservers that are frequently used by spammers, see 3.5.2). However, it has grown into one of the most popular DNSBL services around, and now incorporates data from SPEWS, Spamhaus, plus a list of dial-up services and a few others too. The different lists can be queried individually or all at once, making relays.osirusoft.com a very powerful anti-spam tool.
188.8.131.52 Spamhaus SBL
The Spamhaus SBL (Spamhaus Block List) lists all I.P. addresses belonging to known spammers, spam operations and spam support services. It draws on data from the Spamhaus Project and ROKSO as well as other sources..
The Spam Prevention Early Warning System, or SPEWS, is one of the most controversial DNSBL lists. For one thing the people behind it have chosen to remain anonymous and silent. For another, its policies are surrounded by mystery. It is believed that SPEWS lists spammers and hosts connected with them, presumably based upon some kind of evidence, but the exact criteria they use is uncertain. Mind you, it certainly seems to catch a lot of spam.
SPEWS' website suggests that SPEWS listings are discussed in news.admin.net-abuse.email, which is why you see so many SPEWS-related threads in the newsgroup.
SpamBag.org publish a list of the parts of the Internet controlled by "anti-social elements" (such as those who send large amounts of junk email), as defined by some very detailed criteria layed out on their website. By blocking traffic from machines on this list, providers can protect their customers from such anti-social elements.
184.108.40.206 SpamCop BL
The SpamCop Blocking List DNSBL service is based upon an analysis of the complaints sent through the SpamCop service - the sites that generate the most complaints get listed. While this is a very effective method of stopping lots of spam, it can also result in some alarming mistakes and false-positives, and so this experimental DNSBL list should only be used in caution.
220.127.116.11 SORBS (Spam and Open Relay Blocking System)
SORBS is a DNSBL similar to Osirusoft, witha subtle difference. The block list itself does not scan for open relays, webservers or proxy servers it relies soley on submissions for hosts to test from 'partner' sites. This was done specifically to answer the calls of blocklists 'fighting abuse with abuse'. The internet is a cooperative place, as a third party connecting to a SORBS partner, you authorise a SORBS tester to test your host as a part of your request to connect to the partner's server.
18.104.22.168 Collateral Damage
Most blackhole lists try to be as specific as possible with the exact parts of the Internet that they list. However, sometimes an upstream ISP will move a spamming customer around in their I.P. space, in order to avoid such lists, and it will become necessary to list the entire ISP. However, that ISP will have other, non-spamming customers, who will also suffer the ill-effects of being in the list; these innocents have become collateral damage in the spam wars.
Collateral damage is sadly inescapable, and is directly the fault of those companies who support spammers in this way. Organisations and individuals so affected are advised to find themselves a different, more responsible ISP to escape the collateral damage blackhole.
The analogy of living in a slum neighbourhood is often invoked for those innocent people who become collateral damage, and I find it very appropriate. If you live in a bad part of town, you may find that pizzas won't be delivered after dark, taxis won't hang around, and so forth. Similarly, if you live in a spam-supporting ISP then many other organisations simply won't want anything to do with you. Just like living in a slum, you have two options: either help clean up the neighbourhood (persuade the ISP to stop supporting spam) or move somewhere nicer (find another ISP).
22.214.171.124 I'm not a spammer but I'm being blackholed! How do I fix it?
What has almost certainly happened to you is that your internet provider, or their upstream, has been facilitating spam or spammers in one way or another. Therefore large parts of the Internet have taken the decision to protect themselves from spam by accepting no email from these providers and all their customers.
You are probably an innocent caught in the middle; you're not a spammer but your email is bouncing and you can't contact your friends or your family or your customers. You're entirely justified in feeling very angry about this.
But the many Internet Providers who are shunning your provider are not the right targets for your anger, and neither are the organisations that recommended that your provider be blocked. Instead, you should direct your anger towards your own provider (or their upstream). After all, its their policies, freely decided upon, that have lead to you being cut off from parts of the Internet. If you have a Service Level Agreement with them then you should study it; if your provider is not providing the promised level of service then you may be able to claim compensation or take legal action against them.
If you can persuade your provider to mend their ways, then you will be on the road to becoming free of the blackholings. Alternatively, your only real option is to move to another, less spam-friendly Internet Provider.
You may wonder why the blackholing can't be made specific to the active spammers of the providers, or why just your own I.P. address cannot be removed from the blackhole. Unfortunately, this is not practical, as too many I.S.P.'s have in the past moved their spammers to new I.P. addresses to help them to evade blackholing. To guard against this, the entire I.S.P. in question is generally blackholed.
Your situation is regrettable, and we all wish this wasn't necessary. We feel much sympathy for you, but ultimately we feel more sympathy for the millions of victims of your I.S.P.'s pet spammers.
Occaisionally, you may encounter some problems because your I.S.P. has assigned you an I.P. address that once belonged to a particularly notorious spammer; such addresses often persist in providers' local blocking lists for months or even years after the spammer in question has departed. Since your address is probably present in hundreds or even thousands of such lists, getting it removed from them all will be a next-to-impossible task, so your best course of action in this case would be to ask your I.S.P. for a new I.P. address (and maybe take them to task for selling you damaged goods).
(You may also want to read the answer to question 126.96.36.199, which covers this issue from the other direction.)
7.3 How can an ISP reduce the amount of spam their customers send?
With difficulty. However, experience has shown that there are a few things that can make a difference...
If an ISP has a reputation for dealing with spammers quickly and decisively, many spammers will avoid them. If spammers are dealt with very rapidly indeed, the ISP may be able to shut down a spam-run before it has completed.
An ISP can have a clause in their terms of service that allows them to charge "clean-up fees" to any customers that send spam. Unfortunately, many spammers sign up using stolen credit-card numbers, and in these cases clean-up fees aren't much of a deterrent. It can be messy to collect clean-up fees, too.
An ISP can implement "port 25 filtering" (see 3.5.3 in "Understanding NANAE") to prevent their customers from spamming via open relays. Note that this, however, will prevent their customers from using external mailservers for legitimate reasons too.
An ISP can monitor the email traffic generated by a customer. If a customer who hadn't previously sent more than three or four emails a day suddenly sends a hundred thousand messages, for example, it's a fair bet that he's a spammer and it would be nice if there were systems that would inform the ISP and let them take a closer look.