Security Firms Follow Unwritten Code When Digging Up Dirt On Each Other

A critical vulnerability was spotted Thursday in the anti-virus engine used by Trend Micro's entire line of client, server, and gateway security products, the third such disclosure this month of flaws in major security firms' software.

As in the other two instances with Symantec and F-Secure, the Trend Micro vulnerability was discovered by Internet Security Systems, an Atlanta-based security provider, and revolved around the processing of a compressed file format.

The Trend Micro flaw related to the ARJ file format, which, said ISS, could be used by a hacker to "gain unauthorized access to networks and machines being protected by Trend Micro AntiVirus Library." The affected titles include Trend Micro's Messaging Suite, VirusWall, ScanMail, and PC-cillin lines, among others. A complete list has been posted on Trend Micro's Web site.

An attacker would only have to send an e-mail containing a specially-crafted ARJ file to the target system to compromise the system, added ISS.

Previously, ISS spotted similar vulnerabilities in how Symantec's products handled UPX files and how F-Secure's dealt with ARJ compressed files.

For its part, Trend Micro dubbed the vulnerability "critical," and posted fixes to the affected software on its Web site. Customers were urged to download the updated anti-virus scanning engine from here as soon as possible. Users who don't update manually will receive automatic updates the middle of next week.

While vulnerabilities within security products are rare -- at least in comparison to, say, operating systems such as Windows -- they're not unheard of. And by one analysts' take, they're fair game.

"Within the security community, anytime one finds any vulnerability, it's kosher to make it public if the researcher follows the protocol for responsible disclosure," said John Pescatore, a vice president at Gartner and one of the research firm's security gurus.

In that unwritten protocol, he said, researchers don't publicly disclose a vulnerability until they've alerted the vendor and given it time -- 30 to 45 days at least -- to fix the problem. ISS followed that protocol in all three instances of revealing vulnerabilities in anti-virus firms' products.

"I haven't heard any negative rumblings in the security community about what ISS is doing," said Pescatore. "They've been very above board."

Trend Micro agrees. "ISS is really great to work with," said Bob Hansmann, the product marketing manager for Trend Micro in North America.

According to Pescatore, it's crucial that security software get the once over. "It's even more important than looking for vulnerabilities in Windows or Oracle," he said. "People have a feeling of security when they're using a security product, and if there's a vulnerability in a firewall, for instance, nothing behind that firewall is protected. Everything's exposed."

Trend Micro agreed here, too. "We're actually really happy that people are doing this. The industry needs something like this, not because we need to stir up anything politically [between companies] but because different people tend to look at problems different ways," said Hansmann.

But the practice of one security firm investigating another could be considered inappropriate, said Pescatore, if abused. In the past, various anti-virus firms took potshots at each other, not in public, but by touting the weaknesses in rivals to analysts like Pescatore.

In practice, he said, there's an unwritten rule not to poke in competitors' products, for fear of unleashing the beast. "It's like the old days between the U.S. and the Soviet Union. Neither dared use the Bomb." Likewise, if one vendor picked on a rival, it could only expect that in return.

But the market dynamic is different here, Pescatore said. "ISS doesn't sell anti-virus products, so they're not really direct competitors with Trend Micro, Symantec, and F-Secure. They do get publicity out of this, though."

"Maybe in a year or so, we'll look back and see a pattern, and go, 'okay, that's why ISS was digging into anti-virus code,'" said Hansmann, "but for now, we appreciate what they've done."

ISS itself isn't a stranger to vulnerabilities. About a year ago, the Witty worm exploited an unpatched vulnerability in ISS' BlackICE firewall, infected 10,000 to 50,000 systems, and erased data on some machines.

"If there's one thing I would tweak ISS about," said Pescatore, "it would be that I'm assuming we'll never see anything like the Witty worm in the future if ISS has the time to look for vulnerabilities in other companies' products."

It's not easy to dig up vulnerabilities, said Pescatore: "it takes skill," he said.

"You would have thought they'd been looking at their own products."

ISS did not respond to requests for comment.