New guidelines may pave the way for dozens of UK ISPs (internet service providers) to participate in a University of Cambridge research project on spam.

Spam is estimated to comprise 60 per cent or more of the world's email traffic.

The guidelines concern how ISPs should deal with sensitive issues such as customer privacy and data-protections laws, while cooperating to shut down machines propagating spam, said Martin Hutty, head of public relations for the London Internet Exchange (LINX), a group of around 220 ISPs and network providers.

Just join the dots

When an email is sent from a machine using one ISP to another, both ISPs hold details that can be used to detect spam and locate the machine where the message originated, Hutty said. A user may have been infected with a Trojan horse program, through which a hacker has gained control of the machine and is using it to send spam, he said.

The guidelines will open the door for ISPs that want to participate in spamHINTS, an ongoing research project at the University of Cambridge, Hutty said. Richard Clayton, who holds a doctorate in computer science from Cambridge, heads the research.

"Email is not just a technical problem, but a market failure compounded by regulatory deficiencies," Clayton wrote in a paper outlining spamHINTS.

Traffic analysts

The research project uses traffic analysis rather than content to determine which email is legitimate. Spam, Clayton writes, has characteristics that make it stand out from real mail, even aside from its content.

Spam gets few replies and is often sent out 24 hours a day. It also regional. For example, legitimate traffic flows between the UK and South Korea, but it's uncommon, Clayton writes. Spam tends to consist of a huge number of short messages, while real email is a mixture of sizes and sent in small numbers.

Clayton writes there is very little cooperation between ISPs so far in detecting and reporting spam.

The project, which is funded by LINX and Intel, hopes to tap into LINX's network of ISPs. LINX, a nonprofit organisation that includes members such as Google and the BBC, is primarily known for its peering capabilities, which allow ISPs to connect directly with each other, Hutty said.

LINX gets ready

The direct connection avoids data transit charges for internet traffic carried on other networks, he said.

LINX is enabling its peering infrastructure to produce sFlow data, which consists of packer header information for traffic flowing through its switches. Researchers believe that they will be able to distinguish using the characteristics of the sFLOW traffic between real email and spam, without examining the content, and identify the sending machines.

The end result will be a real-time list of email sources that ISPs can use to investigate misuse. Through heuristic analysis, an ISP should be alerted to odd behaviour, such as if one of their customers starts sending ten times the number of emails as in the previous week.