Firefox Gets A Bad Rap From Shoddy Security Research

Is Firefox currently the Web browser most likely to stick your PC with a dangerous security vulnerability? Only if you believe headlines written by people who really should know better.A great example is a recent article at InternetNews.com. Under a headline that declares, "Firefox Tops Vulnerability List," it offers a distinctly uncritical perspective on a security vendor's press-grabbing claims: Application security vendor Cenzic today released its security trends report for the first half of 2009 application. In it, Cenzic claims that the Mozilla's Firefox browser led the field of Web browsers in terms of total vulnerabilities.

According to Cenzic, Firefox accounted for 44 percent of all browser vulnerabilities reported in the first half of 2009. In contrast, Apple's Safari had 35 percent of all reported browser vulnerability, Microsoft's Internet Explorer was third at 15 percent and Opera had just six percent share.

There is just one problem: Cenzic's figures are based on a methodology so shoddy that it would be funny if it didn't have such serious implications.

Cenzic's research seems to be based on a simple process: Count up the number of security vulnerabilities reported for each browser, convert that into a percentage of the total for all browsers, and alert the press.

As Secunia CTO Thomas Kristensen told The Register, it's a useless approach if one's goal is to get a real grip on a particular browser's actual software security track record: "Other factors need to be taken into account for a proper comparison; this includes the type of vulnerabilities and thus the underlying type of coding errors, the impact of the vulnerabilities, the time it takes the vendor to fix the reported vulnerabilities, how easy it is to update the software thus how quickly the users (learn about and is able to) apply the patches. "One may also want to look at the general design of the product, the efforts invested in improving the code and conducting internal security reviews and quality assurance, the usability with regards to certain security related features, the handling of plug-ins (how easy is it to lure the user into installing untrusted plug-ins) and so on." I'm not making a point here about which browser actually offers better security these days. (For the record, I think that any of the major new releases is far superior to any of the older ones.) I'm suggesting that Cenzic's numbers -- and the resulting media coverage -- are a lousy way to get a legitimate answer to this question.

People who follow these issues closely know better than to accept the media coverage of Cenzic's report at face value. Even when the coverage digs deeper into the meaning behind these numbers, it almost always succumbs to the temptation to lead off with a sensational, and grossly misleading, headline.

This does a grave disservice to readers who are too busy to look more closely. Evaluating software security is a messy, complicated business, but it beats the pants off relying upon simplistic, ham-fisted "research" to serve up easy answers.