UTM perimeter-security devices combine firewalling, antivirus, and intrusion detection and prevention on a single appliance. Many throw in content filtering and antispam, making a compelling argument for one-stop security shopping. Unified threat management, a term coined by IDC, is not a new concept, however: Vendors have tried to bring these processes together for years but were stymied by performance problems. Fortunately, advances in processing, both on the main CPU and in specialized silicon, mean that performance problems can be overcome.

To see how well vendors are tak- ing advantage of new technologies, we tested UTM products in our Syracuse University Real-World Labs, rating each on how well it increases protection without hurting performance. Unfortunately, we found most products wanting. Our scenario was typical: An organization with 5,000 users seeks an edge device that provides firewall, IDS/IPS, antivirus and content filtering. The organization's DMZ network hosts DNS and Web servers that communicate with a back-end Microsoft SQL Server and an SMTP server, which relays mail to an internal Exchange 2000 server (for more details, see "Test Methodology,"). Most traffic is from internal users to the Internet.

We invited 11 vendors to participate. Fortinet, Internet Security Systems, Secure Computing, SonicWall and Symantec accepted our invitation. Check Point Software Technologies, Cisco Systems and Finjan Software said they couldn't ship products in time. Juniper Networks doesn't have an offering that fits, and Astaro and iPolicy Networks did not respond to our invitation.

Secure Enterprise Magazine Current Issue

Advanced Protection?

It's time to puncture the IPS myth: All the problems inherent in intrusion detection are exacerbated by automated prevention. Certainly, there are signatures that flag malicious traffic with a high degree of probability, and you can safely block these packets. Any protocol violation--for example, characters not defined within the HTTP protocol specification, similar to RFC 822--can be intercepted with little risk of a false positive.

Figuring out which other signatures can be safely blocked takes a bit more digging to determine if normal traffic will trigger an alarm.

The vendors in this review take a conservative approach to setting default block policies, and that's appropriate. Each network is different, and an aggressive cookie-cutter stance will likely turn away legitimate traffic. With the exception of ISS's Proventia, changing the default action in the IPS functions of the devices tested was a simple matter of selecting the signature, or in some cases a family of signatures, and setting the action to block. Changing the default IPS setting in Proventia is a multistep process, but it can be done.

Performance Firewall with IPS and Content Filtering
Click to Enlarge

However, in our tests only Proventia properly detected our malicious traffic. The other products failed to detect at least one attack--Fortinet's FortiGate-800 came in last, detecting just two out of five. This is outcome inexcusable: All the vulnerabilities we selected are at least a year old, with publicly available exploit code waltzing through most of these devices and returning a reverse shell on our "attacker's" computer. Moreover, we told all the vendors we'd be using publicly available exploits against servers with known vulnerabilities.

After testing, we shared our results, the tool we used (Metasploit 2.3) and the modules with the vendor participants. Frankly, if we didn't provide the vendors with this information, we aren't convinced they would have added new signatures. In any case, those that didn't fare well will likely have fixes by the time you read this story. We scored protection capabilities, however, based on our initial set of attacks.

Performance Firewall with IPS, Content Filtering and AntiVirus
Click to Enlarge

We then pulled together well-known virus files and sent them over FTP and SMTP with just names, no file extensions, to see if we could sneak any through. Every product except Secure Computing's Sidewinder offered antivirus scanning of FTP traffic, and all the products successfully scanned e-mail attachments.

Signature updates are automatic for the most part, with many of the products able to update manually if necessary. A few of the firewalls required that we install firmware updates and perform reboots manually, but that's par for the course.

We get into the performance versus protection debate a lot. When a stateful packet-filtering firewall wins a review because of better performance, readers with application proxies flame us, saying their firewalls provide superior protection. When an application proxy wins because of better protection, readers with the stateful packet filters rail that those other devices impact network performance. The right answer lies in the middle: We want good performance and strong protection.

Unfortunately, we discovered late that because of a series of errors, the tool we were using to test performance was misreporting results, and the testing scenario we wanted to create was radically different from the one we achieved. So much so that with less than 24 hours until press time, we decided to pull the performance results from this review. We couldn't develop and validate a test on such short notice, and we won't print misleading results. However, we can provide some general observations and post performance charts online.

Reader Poll
Click to Enlarge

UTM Firewall Attacks
Click to Enlarge

Antivirus scanning had a significant impact on overall firewall performance, for two main reasons. Before a firewall can scan files, it must queue them. Then it scans each file in turn and decides to send it, drop it or quarantine it. This process uses memory and introduces a high degree of "burstiness" as files are queued, scanned, and passed or dumped. In addition, virus scanning is CPU- and memory-intensive, and it degrades overall traffic performance. Some firewalls, including the FortiGate and Symantec Gateway Security, let you set antivirus configuration options on a per-rule basis, while others are more global.

Content filtering and IPS functionality generally have lesser, but still appreciable effects on performance. Each vendor defines content filtering differently. For one, it could be as simple as regulating MIME types, or going deep into files to search for key words in Web pages and e-mail. The more specific the content filtering, the slower the overall performance.

Making It Dance

So how configurable are these puppies? We're big fans of granularity--we like to tailor an appliance's protective features to our needs, not the other way around. Fortinet's FortiGate is a model of fine-grained configurability. We could, on a per-rule basis, apply different sets of protection features, such as content filtering and antivirus scanning. That's handy when performance is a concern because you can enable advanced features as needed.

UTM Firewall Features
Click to Enlarge

Once an appliance is processing lots of traffic, the management interface often slows to a crawl, rendering the device unmanageable. If you're brushing off this consideration, you've never tried to manage a firewall that was under DoS attack. We were happy to find that, even while under load, most of the devices remained manageable; the exception was the SonicWall appliance.

The clear winner in our review is ISS's Proventia M50. This $14,890 champ caught all our attacks the first time and had adequate performance and management capabilities, earning it our Tester's Choice award. Symantec's Gateway Security and Secure Computing's Sidewinder G2 battled for the middle of the pack. Both cost more than double what the Proventia will set you back--$36,700 and $35,900, respectively--but each has strengths in application proxies, plus Symantec's offering is augmented with IDS/IPS while the Sidewinder's split DNS and SMTP proxies add a layer of protection to common protocols. Fortinet's and SonicWall's products brought up the rear; we were surprised at how poorly their offerings performed. The FortiGate missed three key attacks, while the SonicWall lacks rivals' policy granularity, has poor logging, underperformed with throughput and missed some key attacks.

Internet Security Systems is a relative newcomer to the UTM field, but its solid experience in intrusion detection and vulnerability assessment have served it well: The Proventia was the only appliance to detect and name all the exploits we used, and really, that's the bottom line. In addition, the wealth of information Proventia provides about attacks and the causes is top-notch. However, its management interface leaves a lot to be desired. Navigating through screens was a challenge, and reporting was middle of the road.

ISS, like other vendors in this review, is conservative in enabling blocking on intrusion-detection rules--a wise stance. The Proventia has a wide variety of actions that can result from alerts, but the most common are blocking or resetting the connection and dropping the packet. Tuning IDS/IPS features--a requirement in nearly all IT shops--was needlessly complicated. We could tune default actions by entering adjustments on a per-signature or event-family basis through an advanced tab widget. For a few exceptions, this wouldn't be a problem, but for making wide-scale changes, it becomes another rule set to manage. ISS will make changes for you, but then you have to keep going back to the mother ship--not a scalable solution.