News
News
8/25/2006
06:20 PM
Connect Directly
RSS
E-Mail
50%
50%

Anti-Spyware Vendors Mad About Consumer Reports Test Methods

Vendors including Microsoft and Sunbelt Software say the consumer magazine's test is bogus because it doesn't take into account how security software detects and removes threats.

Consumer Reports defended its actions by saying it was the best way to test anti-virus software against "novel threats not identified on signature lists."

McAfee and others, including Abrams, now the director of technical education at Eset, a Slovakian security company, rejected that claim. Instead, they pointed out that "retrospective" testing -- where an anti-virus product is installed but then not allowed to update its definitions for several weeks or months -- is a more realistic way to test how software handles new threats. Products can be objectively ranked, said Abrams, when new-found malware is tossed at out-of-date definition files. The practice quickly shows how well each anti-virus program sniffs out new attacks using its behavior-based tools.

"I thought that their anti-virus testing was bogus and useless," said Abrams. "But their anti-spyware testing was worse."

Consumer Reports did not respond to a call for comment, but in a letter from Dean Gallea, its test program leader, to Eckelberry that that latter shared with TechWeb, the publication didn't sound like it was about to change its mind, or its anti-virus and anti-spyware conclusions.

"Thanks for your insights on the use of behavior simulation to test the performance of anti-spyware programs. We believe we understand your concerns, however we chose this approach because we felt it best captured the flexibility of the software," Gallea wrote.

"We are constantly re-evaluating our test program, and will take these and other considerations into account in future tests," he added.

"I grabbed a copy [of Consumer Reports]," said Abrams. "On the cover it said 'sleeping pills, the facts they don't tell you,' or something along those lines. But if they do such an incredibly bad job in testing consumer anti-virus and anti-spyware, how could I ever trust them with something medical related? I was completely dumbfounded by the whole thing," he added.

"I am happy that NOD32 [Eset's anti-virus title] was not tested. There is no honor in claiming that you were number one in a worthless test. It would be like being rated best food by a critic who has no taste buds and doesn't understand nutritional content."

Previous
2 of 2
Next
Comment  | 
Print  | 
More Insights
The Business of Going Digital
The Business of Going Digital
Digital business isn't about changing code; it's about changing what legacy sales, distribution, customer service, and product groups do in the new digital age. It's about bringing big data analytics, mobile, social, marketing automation, cloud computing, and the app economy together to launch new products and services. We're seeing new titles in this digital revolution, new responsibilities, new business models, and major shifts in technology spending.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Tech Digest - July10, 2014
When selecting servers to support analytics, consider data center capacity, storage, and computational intensity.
Flash Poll
Video
Slideshows
Twitter Feed
InformationWeek Radio
Archived InformationWeek Radio
Join InformationWeek’s Lorna Garey and Mike Healey, president of Yeoman Technology Group, an engineering and research firm focused on maximizing technology investments, to discuss the right way to go digital.
Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.