Is Recurrent Pattern Detection the answer to spam?

Alan Thwaits at eChannelLine discusses RPD (Recurrent Pattern Detection) and its potential to stopping spam (or I should say "reducing" spam since the goal on their part is to reduce it by 95%).

The report identified three main criteria for anti-spam solutions: the need to block 94 to 95 per cent of spam; the ability to trace spam in the first few minutes of an outbreak; and the ability to block spam regardless of the language or dialects used.

Commtouch's RPD technology wins on all three counts, according to Oren Drori, the company's director of product marketing.

"Instead of looking at the content of each message, like other anti-spam technologies, RPD looks at the Internet as a whole," he explained. "It looks for recurrences of patterns in huge amounts of mail. If you can trace the repetitive pattern, you can block the spam."

Commtouch's anti-spam detection center monitors approximately 30 to 40 million messages a day, which Drori said is a representative sample of the Internet itself.

It sounds as if it works by monitoring a distributed selection of mail off of the net, regardless of location or language (I would imagine ideally spanning many countries and areas), for patterns that show up in N number of messages. If that N value is sufficiently large, then it marks that signature as "spam" and then all clients that see that will react accordingly.

I could be misunderstanding the technology and/or the methods used, but it sounds an awful lot like CloudMark's SpamNet, which in my experience is slow to catch on.
Perhaps this is different enough to actually work quite well, I don't know.

Posted by Eric at May 9, 2004 10:17 PM | TrackBack