McAfee Update Breaks Hundreds Of Apps

For over five hours Friday, McAfee's anti-virus software erroneously flagged hundreds of legitimate executables as a malicious virus, leading some customers to quarantine or delete the offending files and render applications such as Microsoft Excel inoperative.

An error in McAfee's daily virus definition file (dubbed "DAT") identified the files as W95/CTX, a virus first discovered in 2004. All editions of McAfee's on-demand-scanning products, including both the enterprise and consumer versions of VirusScan, were affected.

Among the legitimate files painted as malware were Microsoft's Excel spreadsheet, Adobe's Flash, the Google Toolbar installer, several Adaptec drivers, and parts of Sun Microsystems' Java Runtime Environment. The list that McAfee posted of the affected files numbers more than 330, but even so, the SANS Institute's Internet Storm Center called it incomplete.

"It doesn't include any of the Oracle binaries that have been reported to be affected by some of our readers," one of the Storm Center's analysts wrote on the site Sunday.

Depending on how users had configured VirusScan, the harmless files were either quarantined to a special folder or deleted. In either case, applications were broken as files were moved or erased from hard drives.

The flawed DAT went out at 10:35 a.m. PST Friday, said Joe Telafici, director of operations at McAfee's AVERT Labs. "About two hours later, we started getting reports of large numbers of files identified as W95/CTX," he said.

McAfee pushed out a corrected DAT a couple hours after that, at 3:28 p.m. PST.

By then, however, it was too late for some McAfee users.

Customers flooded the company's message forums with questions and tales of broken applications. "I tried to open Excel, and it tries to install itself again, then fails," wrote a poster identified only as "Waterlily." "I need to use Excel, what should I do?"

"So what do you do if instead of quarantining those files, you deleted them?" asked "Bethany." "I bet I'm just screwed."

Quarantined files could be restored, said Telafici, once the corrected DAT was downloaded and installed, but deleted files were another matter. On its Web site, McAfee recommended going to a backup or using Windows XP's System Restore feature to roll back the machine to a point before the flawed DAT.

"We're still looking at what we can do for customers," said Telafici. He wouldn't quantify how many users might have deleted files, and only said that McAfee was working with "some."

However, McAfee has come up with tools to move quarantined files on enterprise machines back to their proper places. The tools have not been posted to its Web site, but will instead be provided to business users through offline support channels.

Such "false positives" are much more common in spam than in virus detection, but they occasionally happen, Telafici said.

"But I've never seen anything on this scale," he admitted.

"False positives are actually very common," added Richard Stiennon, chief research analyst at Michigan-based IT-Harvest. "They're particularly common for programs that aren't widely distributed, like some game."

Most of the time the mistaken identity occurs when a security researcher finds a malicious file and tags its filename as belonging to a virus or worm, but doesn't realize that the same filename has been used by a legitimate program.

This should have been caught by McAfee's quality control process, Stiennon said, noting that many of the files netted by VirusScan were commonly-known executables.

McAfee's Telafici acknowledged as much.

"This was a combination of unusual circumstances, Telafici said in explaining what happened. "There was one byte off in a signature, and there was a hole in our testing process."