The prospect of a world without email is something that most of us would prefer not to think about.
For millions of staff at tens of thousands of companies, the working day begins and ends with email, and occupies a significant portion of their time in between. Email browsers such as Microsoft's Outlook have grown to become the de facto user interface to many of our key applications; mobile email devices, such as the RIM BlackBerry are key elements in the mobile worker's armoury; and personal email-based filing systems are now thought to contain as much as 80% of some companies' unstructured data assets.
Email, indeed, has grown to be an indispensable part of the corporate information infrastructure, "displacing the telephone in most organizations as the primary mission-critical communications channel," according to Jeff Smith, CEO of email filtering software vendor Tumbleweed Communications.
Unfortunately email was not meant to fulfill such a pivotal role. "It was never designed to be secure. It was designed to be easy to use and to be anonymous," says Smith.
Today, these twin characteristics, which have made it so successful, are now at the root of a problem that is threatening the future of corporate email: the epidemic of unsolicited email that has come to be known disparagingly as ‘spam'.
Certainly, on an individual basis it is easy to be sanguine about spam. Although constant invitations to view pornography, consolidate our debts, improve our sex lives, get rich quick or find God can be irritating, and sometimes offensive, most people can recognize spam when they see it and are content to hold down the delete key for few seconds before getting on with their work.
However, as far as a growing number of IT professionals are concerned, spam is not something that can be dismissed so casually. It is costing their companies money, and it is threatening the security of their IT systems.
In 2004, according to IBM's Security Threats and Attacks Trend Report, around 70% of the world's Internet traffic was accounted for by an average daily deluge of 24.8 billion unsolicited email messages. The bulk of this vast quantity of electronic junk is delivered, processed and ultimately stored using network capacity, email servers and disk drives owned and paid for by businesses – and that is not a trivial expense.
In 2002, when spam email was thought to account for just 40% of all email (compared with up to 80% today), researchers at the University of Baltimore estimated that hijacked use of these resources, and the productivity lost by the need for workers to identify and bin spam, cost US corporations $8.9 billion a year.
This cost to business is now almost certainly significantly higher. Not only has the volume of spam doubled in the past three years, so too has the incidence of malware and security exploits that can be associated with spam, and the operators that produce it.
Spam has long provided a useful medium for malware, such as Trojan horses and spyware, allowing such software to hitch a lift through the firewalls and anti-virus filters of unsuspecting companies. Indeed in Germany, for instance, managed security services provider Ubizen says that as many as 1 in 12 emails blocked by its systems last year was carrying some kind of virus.
Such spam-borne malware attacks are well understood, and relatively easy to counter using established email filtering products. A more worrying trend is the rise in so-called ‘dark traffic' exploits such as denial of service (DoS) or directory harvesting attacks (DHA).
DHA and DoS are not necessarily spam in the classic sense. Some DoS attacks do use fake emails to bombard targeted email servers into submission – for the purpose of industrial sabotage, blackmail or, simply, malice – but they do not necessarily need an email payload to be effective.
Instead, by exploiting the ‘trust' that companies typically place in the so-called Port 25 channel that normally allows SMTP (simple mail transfer protocol) email traffic to pass through the corporate firewall, exploits such as a DHA can provide a spammer with relatively unhindered access to a variety of information sources. Typically, this will mean taking addresses from the email directory as a precursor to a spamming campaign. But, as single sign-on products such as Active Directory become more prevalent, network, systems and personal security information can become the target of the same attack.
These twin escalations in both the scale and the threat of the spam problem may overwhelm some companies, says James Kay, chief technology officer of managed email service provider Black Spider Technologies.
"By the time most of our customers come to us, more than 50% of their email is already spam and some are at or closer to 80%. As the amount of email they need to process is doubling, so they have twice as much to do, and so they employ twice as many staff to do the work. I have seen one organisation with a team of half a dozen network administrators working full-time stripping out spam. They're quite expensive people, so a team like that probably costs £200,000 to £300,000 per year," said Kay
The fact that some organisations are being forced to divert such scarce and expensive resources to an activity that offers little or no direct benefit to their business effectiveness demonstrates not only how severe the spam problem is, but also how ineffective traditional measures have become.
First generation spam filtering products, which identify and block spam email on the basis of message content alone, have always been difficult to employ successfully and can certainly no longer be relied on in isolation. Spammers now use automated authoring tools that avoid or misspell commonly targeted words and phrases, allowing their products to pass the filtering test, but there is no guarantee that legitimate and potentially critical mail will do the same.
In the healthcare sector, for example, "breast, penis and Viagra are all pretty universally used terms" that are as likely to be weeded out of legitimate mail as junk, says the CEO of anti-spam software and services provider Proofpoint, Gary Steel. Whilst even in sectors blessed with more prosaic jargon, such as financial services, filtering for terms that would normally be considered offensive is not really an option.
As Michael Colao, director of information management at investment bank Dresdner Kleinwort Wasserstein bank points out, the abusive email sent by a passionately disappointed key customer is exactly the kind of mail no company can afford to lose.
As well as highlighting how dangerous it is to rely on a single, relatively crude method of spam filtering, Steel and Colao's examples also make the point that the effectiveness of anti-spam products must be measured as much by how little legitimate mail is intercepted, as by how much junk is blocked.
Technical experts, such as Black Spider's Kay, believe that some companies have not yet properly appreciated this fact, and are still focused on finding a ‘total solution' that blocks all spam. In fact, he says, "anyone who tells you they can block 100% of spam shouldn't be taken seriously. The only way to block all spam is to block all email."
Clearly, some spam is infinitely better than no email, and even an irritatingly high proportion of spam is a better option than a high ratio of false-positive identifications that filter important messages into the trash can. Predictably, most vendors recognise this, and now claim to be able to achieve the right balance between a high true-positive blocking and low false-positive rate. Indeed, in practice, says Matt Cain, an analyst at market research group Meta (now part of Gartner), in terms of blocking performance there is little to distinguish between them. Almost all vendors now claim to be able to block 95% of spam, and "false positive rates are [also] relatively consistent, ranging from a ratio of one in 10,000, to one in 1 million," he says.
This is generally good news for customers, although it does mean that selecting between prospective anti-spam vendors requires them to look at a variety of features beyond merely their ability to block spam.
Email Hygiene, a vendor comparison report published by Meta in November 2004, concludes that blocking performance is only one of 12 factors that customers should consider when selecting an anti-spam solution. In fact, Meta's Cain believes that although the spam problem is usually the primary driver behind most companies' email security investments, the best return on such an investment is likely to come from solutions that take a more holistic approach to the effective management of email.
Thus, whilst the ability of products to block spam, weed out viruses and protect against message transfer agent (MTA) and SMTP-based attacks is a central consideration in any buying decision, customers should also look carefully at issues such as email security and encryption; outbound as well as inbound content filtering (to protect against malicious or accidental dissemination of confidential information); email management and reporting tools; any particular vendor's native technical capabilities (for example, do they develop their own anti-spam technologies, or do they primarily license them from others?); and, of course, cost of ownership.
Sorting through such a complex mixture of unfamiliar differentiating criteria is a daunting task. On the technology front, for instance, anti-spam products vendors now typically offer a dizzying array of different spam filtering techniques that, in combination, have the potential to provide a very fine-grained and accurate means of separating legitimate mail from spam.
Barracuda Networks, for example, boasts of its layered approach to spam filtering. This begins with network layer scrutiny against mass mailings that may be DoS attacks, and then escalates through nine more layers that include testing against lists of known spammers' IP addresses, two separate virus checks, a layer of user-defined testing parameters, and a final analysis that reviews an email's performance against all the previous tests.
Barracuda is not the only company to offer such comprehensive filtering, but the point is, says the company's VP for EMEA, Paul Thackeray, that its spam firewalls ensure that it can be done cost effectively, with minimal impact on network performance.
An entry-level Barracuda M200 spam firewall, says Thackeray, can be user-installed in a few hours, and over the next several days can be remotely tuned by Barracuda to support up to 1,000 email users, and cope with inbound traffic of up to 1 million emails per day. The device costs under £1,400, and as its filtering algorithms are constantly updated for an annual fee of around £300, ongoing costs should be minimal.
Appliance vendors like Barracuda and Borderware (which was recently acquired by 3Com) argue that for smaller companies their device-lead approach to email hygiene is easier and cheaper than software-based solutions, and can be more reliably scaled-up to meet the very high filtering demands required by larger companies. However, their software and managed service-oriented competitors (whose licence fee-based charging models can seem comparatively expensive on a per user basis) claim that email management is too complex to be left entirely to a relatively ‘dumb' box – however sophisticated its capabilities appear to be.
Black Spider's Kay, for example, argues that his company's outsourced managed service not only takes his companies' email servers out of the direct firing line of spam and hacker attacks, it also provides protection that is individually tuned to each customer's requirements. At £2.50 to £3.50 per user per month, this may compare badly with the annual cost of an appliance upgrade subscription, but it is still a small price for an effectively managed mission-critical email environment.
Although vendors will always be able to find failings in their competitors' offerings, in one respect they are united. Spam is not going away. Even though the war on spam has only just begun it is already clear that it is a war that must be fought on many fronts – using a variety of techniques. Companies that do not grasp this fact, and commit to a multi-layered stance against spam will lose the fight, and with it the value of the most effective and low-cost corporate communications path ever: electronic mail.