Which operating system, embedded in more than 80% of enterprise IT environments, represents one of the fastest-growing hacker targets and potentially the most-devastating information-security vulnerability? Hint: It ain't Windows.

Cisco Systems' Internetwork Operating System now sits at the center of the information security vortex. Because IOS controls the routers that underpin most business networks as well as the Internet, anyone exploiting its flaws stands to wreak havoc on those networks and maybe even reach into the computer systems and databases connected to them. IOS is a highly sophisticated piece of software, but--as with Microsoft's Windows--that's a double-edged proposition. Software complexity can be a hacker's best friend.

10 The number of IOS security advisories Cisco has issued in the last two years

Cisco is working hard to better shield its routers and other network equipment from the risks, but there are reasons to believe Cisco security will become a bigger problem before it gets better. The sheer amount of Cisco equipment installed, the many versions of IOS involved, the difficulties of upgrading that software, and the IOS vulnerabilities already out there or yet to be discovered present a major challenge to network administrators and security professionals.

Just last week, Cisco issued a security advisory for a serious IOS "heap-overflow" vulnerability that could let hackers get control of routers and switches running certain versions of the software. Cisco said it's not aware of any "active exploitation" of the vulnerability, which will give customers at least short-term comfort. But Cisco notes that successful exploitations of similar vulnerabilities in the past have resulted in denial of service when the exploit caused a router to crash and reload. "In the event of successful remote code execution," Cisco warns, "device integrity will have been completely compromised."

Proof Positive

This particular problem first came to light in July when information-security researcher Michael Lynn took the podium at the Black Hat conference with a presentation that proved hackers actually could take over IOS, not just shut down Cisco routers. Lynn, who'd been studying IOS code while working for Internet Security Systems Inc., dispelled the widely held notion that it was impossible to exploit IOS buffer overflows to take control of Cisco equipment. He revealed an attack vector in IOS version 12.3(5b) running in IPv6 environments that could be used by hackers to gain control of network traffic; remotely examine, or "sniff," packet content; modify traffic; and break weak encryption. Lynn went out on a limb to share what he knew, resigning from his job at ISS to make the Black Hat presentation, rather than quiet down. Cisco later obtained a court order to shut him up (see story, "The 'Unthinkable' Becomes Possible").

Why the fracas? The worry is that cyberpunks could apply what Lynn revealed to gain access to passwords, sneak into networks, or redirect traffic to spoof sites that steal identities. A single point of failure becomes a gateway to all kinds of trouble. "Our networks are becoming very gray with all of the different types of access," says Dan Lukas, lead security architect for Aurora Health Care in Wisconsin, a not-for-profit health-care network with 14 hospitals, 150 clinics, and more than 200 pharmacies. "You don't have any boundary anymore."

On the prospect of future vulnerabilities for router infrastructures, Lukas says, "I see it coming."

Cisco took issue with Lynn's public disclosure, saying it was waiting until it had patches that could be applied to all IOS versions before making an announcement, but it doesn't deny the severity of the vulnerability. "Remote code execution is one of the highest impacts you have, because once you do that, you can do anything on the device," acknowledges Mike Caudill, product security incident manager for Cisco's Product Security Incident Response Team.

$148M What Cisco spent in the past year to buy three IT-security companies

Cracking the IOS code to do this kind of harm is harder than hijacking Windows, but Cisco has found it increasingly difficult not only to plug the operating system's security holes but also to get customers to update to the newest versions. The customer heel-dragging is caused by IOS's complexity and by the work involved in upgrading. Over time, Cisco has built many security capabilities into IOS, which is one way to create a secure network, but not the only way. 3Com Corp., for example, lets its Tipping Point intrusion-prevention appliances handle much of the security workload rather than burdening its network operating system. Cisco, meanwhile, keeps layering on more defenses. This month, IOS version 12.4(4)T debuts with deep-packet inspection pattern-matching and filtering capabilities to help companies respond to virus outbreaks.

Complex Upgrades

One consequence of making IOS better is that it also keeps getting bigger. "IOS has become large, monolithic, and bloated with features and functions," says Forrester Research analyst Robert Whiteley. That makes network administrators reluctant to upgrade to the latest version because of the testing and implementation work involved.

Cisco's routers and switches have been built on a variety of processors, including PowerPC and RISC-based chips. As a result, there isn't a single IOS code base that runs on all Cisco products. "That's why Cisco has so many different IOS code trains," says Greg Shipley, chief technology officer of security consulting firm Neohapsis. Juniper Networks Inc., by comparison, has a standardized operating system code base across all of its routers. "It's not that Juniper has never had security problems, but their routers are easier to upgrade than Cisco's," Shipley says.

Here's how it plays out among Cisco's customers. Aurora Health Care uses about 250 Cisco routers, and patching them requires replacing each IOS version with an updated version, then rebooting the system and making sure that the improved IOS doesn't interfere with network cards or other network devices plugged into the router or switch. "If it's not broken, we don't try to fix it," Lukas says. "We can run the same code on a router for a year."