News and Reviews

SpamAssassin vs. M-Switch Anti-Spam performance tests highlight hidden costs of open source anti-spam.

Whilst increasing numbers of organisations are realising the benefits of server-level anti-spam solutions, the lack of transparent benchmarking tests are hampering assessments of 'free' Open Source solutions against their commercial competitors.

(LONDON. - April 27, 2004) - In a whitepaper released today, Isode assess their own anti-spam solution 'M-Switch Anti-Spam' against the leading open source package, SpamAssassin. The results highlight the significant performance failures of SpamAssassin as well as the hidden costs inherent in adopting open source anti-spam software.

M-Switch Anti Spam

M-Switch Anti-Spam is a comprehensive server-lever defence against spam. Using a range of advanced filtering techniques M-Switch provides excellent spam detection and word-blocking based content filtering as well as management interfaces that enable easy system configuration, management and monitoring of anti-spam behaviour, spam categorisation and the ability to choose spam processing actions and behaviour for each user.

M-Switch includes SMTP message switching and as such can be deployed as a complete solution by service providers. For the purposes of benchmarking, SpamAssassin, which does not include an SMTP message switch, was paired with Postfix, one of the fastest free message switches available.

Performance Gap

Spam removal was tested using a collection of 1650 recent messages taken from a spam trap email box. With both products operating at their default settings on identical platforms, SpamAssassin missed 44% of spam compared to 5% for M-Switch.

With an average message size of 10kbytes, the SpamAssassin/Postfix solution was additionally able to process only 8 messages per second, compared to 58 per second for M-Switch.

Another key measure of the effectiveness of an anti-spam solution is its rate of 'false positives', the percentage of real emails incorrectly treated as spam. Working with a batch of 1200 messages, M-Switch returned a false positive rate of 0.15% whilst SpamAssassin (0.9%) incorrectly treated nearly 1 in 100 real and possibly vital email messages as if they were spam.

No Free Lunch

"The hard fact is that there is no such thing as free enterprise software" commented Isode CEO, Steve Kille.

"Quite apart from the performance issues that our testing has highlighted, users of open source software incur considerable installation and configuration costs as well as the ongoing operational and support costs that arise from having to provide their own local support for a product with no vendor."

"We’ve worked hard to perform these trials in a neutral manner and explain how we obtained our results. The detail we’re providing in our whitepaper should enable others to duplicate our tests and we encourage them to do so."

About Isode

Isode is a software product company, originally established in 1992. Its M-Vault (LDAP/X.500 Directory Server) and M-Switch (SMTP/X.400 Message Switch) products are used around the world by ISPs, Multinational Corporations, Government Departments, and Universities inluding BT Ignite, Federal Bridge Certification Authority, Novis Telecom, EDS, PTM.com, Queens University Belfast, Steria Limited, Her Majestys Government Computing Centre (HMGCC), Leeds Learning and The Government of Alberta.

About Steve Kille

Steve Kille founded Isode in 1992. A well known speaker and recognized industry visionary, Steve has twenty years of experience with messaging, directory, security, and e-commerce, and has been responsible for a range of widely deployed products and standards. He has written over forty RFCs (Internet Standards), and is one of the authors of LDAP (Lightweight Directory Access Protocol)

Contact Information

Isode
Lydia Steyn
+44 (20) 8783 0203
lydia.steyn@isode.com

Wednesday, December 17th, 2003

Section Contents

Press