Review: Web Application Firewalls

Think you know what Web sites are running on your servers? So did we. Then we started testing Web application firewalls and saw requests coming in for a site we didn't recognize--and which, by the way, was vulnerable. We assumed a vendor had left old data on an appliance under test, but all the vendors we asked insisted this was not the case. So we did an NSLOOKUP, and lo and behold, discovered one of our programmers was running a nonprofit Web site on our development server.

Heed the voice of experience--if you want to know exactly what's going on with your Web servers, a Web application firewall, or WAF, is worth every penny. Available in software or appliance form, WAFs work at the application layer, using deep-packet inspection to reveal the inner workings of Web applications while thwarting attacks made possible by insecure programming.

We invited WAF appliance vendors to send gear to our Syracuse University Real-World Labs®. We specified that products must inspect HTTP traffic and make decisions at the application layer to detect and stop common Web attacks, including SQL injection, buffer overflows, form-field manipulation, session hijacking, path traversal and forceful browsing.

Products from Breach Security, Citrix Systems, F5 Networks, Imperva, NetContinuum and Protegrity met our criteria, and the vendors agreed to participate. But Citrix bowed out early, saying it lacked the technical resources required to support our testing, and after an initial evaluation of Protegrity's appliance, we recommended that the vendor pull out of the review because its policy builder and reporting interface were severely lacking in detail. During initial tests (with Protegrity engineers in-house), the appliance kept blocking traffic to a valid URL. Logs gave no explanation as to why the site was blocked, and company reps spent a couple hours trying to solve the problem--to no avail.

And Then There Were Four

Four appliances were left standing, and we were impressed with all of them. Each took a slightly different approach to Web application protection. BreachGate runs offline--the WAF links to the Web server through a hub or mirrored port on a switch. Breach says this eliminates the device as a point of failure or latency but, because traffic doesn't go through it, blocking when attacks are detected is a hit-or-miss proposition in low-latency environments.

Imperva's SecureSphere runs as a bridge; we placed the WAF directly in front of our Web server and passed all traffic through it. It's easy to set up, but it could be a point of failure or performance bottleneck. Although NetContinuum's NC-1100-AF and F5's Big-IP with Application Security Module can run in several modes, we set them up as reverse proxies for our tests; this is the most secure way to use the appliances, and our vendor reps told us it is also the most common. In reverse-proxy mode, the WAF acts as a middleman between client browser and Web server. Setup was more difficult and there was some latency, but in return we got tight security. Note that the NC-1100-AF is the only device we tested that doesn't have the option to fail open.

Some of our entries have their roots in the conventional firewall world, while others evolved from intrusion-detection devices. BreachGate and SecureSphere were built from the ground up as WAFs, while the NC-1100-AF's roots are in application security and traffic management, and Big-IP from the load-balancing realm. Organizations looking for a WAF integrated with load balancing, SSL acceleration or caching should consider the F5 or NetContinuum devices.

Although we were impressed with all four WAFs, don't be lulled into thinking they'll eliminate every vulnerability. They don't lessen the importance of testing and securing code, and they certainly don't replace patching. However, they are impressive in their ability to identify and build rules based on the most common types of Web application attacks. Even as they continue to merge with other services, such as caching servers, Web acceleration and load balancers, we expect WAFs to become an integral part of most enterprise networks. In fact, all leading application delivery controllers will include a WAF function by 2007, according to Gartner. And by 2008, 80 percent of shops with WAF and application-acceleration devices will be using a combination device.

From Learning to Adaptation

All the devices had some type of learning mechanism built in. The units from Imperva and Breach were pure learning-based firewalls, while the appliances from F5 and NetContinuum had more limited learning capabilities. In learning mode, the WAF is typically set up to monitor for, but not block, attacks. Depending on your traffic patterns, it could take days or weeks for an appliance to reach the adaptation phase, where it has enough knowledge about Web site traffic to dynamically build a set of policies. These policies are based on input types, traffic patterns, and in Breach's case, HTTP responses. Overall, the WAFs did an excellent job in learning our test environment, but we still needed to adjust policies based on our knowledge of applications.

Our testing scenario placed each WAF in front of a Web server farm, where it monitored HTTP traffic and optionally inspected outgoing responses for acceptable data. The products can "learn" or not, and we didn't require that they terminate SSL traffic. We did expect them to block common attack vectors, including form manipulation, cookie tampering, SQL injection, session stealing, forceful browsing and other Layer 7 attacks .

Based on this scenario and our Web administration experience, we defined five main areas to judge each WAF: policy development, management and configuration, reporting and troubleshooting, accuracy, and price.

With policy development, we want an appliance that provides a high degree of flexibility in configuring and tuning policies, regardless of whether the appliance is learning-based or not. To us, policy development is the heart of a WAF. Although the BreachGate was the easiest to tune, it wasn't the most robust. SecureSphere slightly outdid the NC-1100-AF and Big-IP, thanks to the classification and signature details SecureSphere provided. Specifically, we like the way SecureSphere breaks policies/signatures into classes, such as firewall rules, signature rules and protocol-violation rules. In addition, for each signature the SecureSphere showed a summary of the signature, attack information, affected systems, outside Web references and accuracy information, including details on false positives.

The NC-1100-AF's policy manager is an expression builder that helped us build and edit policies, while the Big-IP provides a built-in crawler and browser recorder, useful for configuring policies.

Aside from policy development, Web administrators will spend a great deal of time managing and configuring a WAF, from defining role-based administration to setting up global firewall settings. Although we found the management and configuration tools provided by BreachGate and SecureSphere easy to use and intuitive, we gave the edge to the Big-IP and NC-1100-AF, both of which offer more robust toolsets. All the products have some type of role-based administration, but the Big-IP, SecureSphere and NC-1100-AF add Web services controls, such as SOAP security options. However, only the Big-IP and NC-1100-AF can load balance and cache; they gave us impressive control by offering features such as negative regular expressions, where requests matching those expressions were blocked.

WAF troubleshooting and reporting are key to providing a secure environment with the fewest possible false positives. In addition, reporting will help prove the worthiness of the product and, in some cases, is crucial to industry compliance. We saw major discrepancies in this area.

All but the BreachGate have intuitive, task-based help built into every part of the management interface--extremely useful when setting up and configuring the appliances. SecureSphere and BreachGate allowed us to group all events in an alert viewer so we didn't have to scroll through thousands of similar vulnerabilities, a la Windows Event Viewer. In addition, the SecureSphere and Big-IP let us see full packet requests, helpful when retracing a potential attack. SecureSphere really stepped up to the plate with its large and comprehensive set of built-in reports. We could export all SecureSphere reports into Crystal Reports, PDF or RTF formats. In contrast, Big-IP's built-in reports were of very poor quality, and we couldn't export BreachGate log data into .TXT files--every time we tried, we got errors--and this made it difficult to perform analysis.

Rather than running potentially unreliable performance tests, we focused on the accuracy of our vulnerability tests. Using Watchfire's AppScan 6.0 we fired about 8,000 attacks at each appliance and were more than impressed--nearly all of the attacks were blocked, and those not blocked were less critical vulnerabilities. For example, Watchfire considers sensitive information in HTML comments a vulnerability, yet none of the appliances stripped comments from pages. Although we agree some comments could constitute vulnerabilities, we don't believe WAFs should be expected to do natural-language processing on HTML comments. Overall, we give the Big-IP and NC-1100-AF the edge on accuracy based on their positive security models.

Finally, we evaluated price. All the WAFs were competitive here; you can expect to pay around $30,000 for a basic model. The Big-IP that F5 sent us is priced at $65,000, but the company recently released a new appliance that it says does not include load balancing and is not as robust as the one tested, but is priced at $28,000.

Although we wouldn't kick any of the appliances out of our lab, Imperva's SecureSphere is our Editor's Choice. We're partial to learning-based firewalls, and the SecureSphere came with a set of built-in signatures that we could update and modify. Its clean, easy-to-use interface was intuitive and provided for comprehensive policy development, and we liked seeing specific signatures that educated us on attack information and severity.

F5's Big-IP was the most complex device to work with, due to its traffic-management roots. F5, like NetContinuum, uses a positive security model, but Big-IP setup was easier thanks to its spidering tool. One highlight is how F5 assigns a tracking number to each blocked request. This is a boon for both admins and end users, as everyone is given a reference as to what was blocked and admins can dig deeper to find out why.

The NC-1100-AF uses a positive security model and is not learning-based. Some may appreciate this model, which allows in only approved traffic, but we found setup too time-consuming. Although we don't mind trading some time for tighter security, we're not convinced that a positive model adds protections that cannot be found using a learning-based WAF in much less time.

Breach's was the easiest appliance to run, in terms of its user interface. However, the BreachGate lacks some advanced features, such as the ability to modify signatures, do in-depth reporting and work in multiple modes, as we could in NC-1100-AF. Breach is newer to this market than rivals but has the foundation to make it a force to be reckoned with in future releases.

Jeffrey H. Rubin is a senior instructor with the School of Information Studies at Syracuse University and president of Internet Consulting Services. Send your comments on this article to him at [email protected].

Ravind Budhiraja is a Web administrator for the College of Law at Syracuse University and a consultant with Internet Consulting Services. Send your comments on this article to him at [email protected].

How We Tested

We invited technicians from each of the four vendors to our Syracuse University Real-World Labs®, where they assisted us in installing and configuring each appliance. We were impressed that installation took as little as five minutes (Imperva) and no longer than a couple of hours (F5), depending on whether the product required network reconfiguration (proxy-based) or ran as a bridge or offline. The WAFs were ready to use after initial setup.

We placed each WAF in front of our development Web server. Our Web servers are dual-Pentium 6, 993-MHz Dell PowerEdge 2250s with 1 GB of RAM, running Windows 2003 Enterprise Edition, Internet Information Server (IIS) 6.0 and a SQL 2000 database server. Our development Web server hosts more than 50 dynamic Web sites, which vary in size from 20 pages to 5,000 pages, all built in either ASP or ASP.NET and most integrating with the SQL database.

Some of the appliances are learning-based, so we used load-generating scripts to generate enough uniform traffic to let us switch to an active protection mode. In a nontest setting, the traffic generated through the learning process would have been more indicative of "real-world" traffic. However, we were more interested in seeing the products adapt, to determine how adaptation affects vulnerability scanning.

Watchfire generously supplied us with a copy of AppScan 6.0 to assist in our vulnerability testing. We reviewed AppScan 5.0 in September 2005 and were not overly impressed. However, all our complaints seem to have been answered in version 6.0, so we were comfortable using the product for this review. Watchfire also provided a demo banking application, named AcmeHackme. The AcmeHackme Web site was developed using C# and runs within Microsoft's .Net framework. AcmeHackme is a live, working, banking Web application chock full of vulnerabilities, including forceful browsing, parameter tampering, XSS, SQL injections, cookie poisoning and SOAP/XML manipulation.

We first ran the vulnerability scanner against the AcmeHackme Web site with no WAF in place. There were nearly 8,000 vulnerabilities in the site, the majority of which were exploited. We then put each WAF in place and ran a full scan. We were impressed with how well each appliance blocked the majority of the attacks launched at the site. Out of the box, NetContinuum performed best, but after Imperva and Breach were able to fully learn the Web site, they provided excellent results, as did F5.

We had planned to run performance tests using WebAvalanche and Reflector. We were interested in how much latency existed with the algorithm that each appliance uses to inspect a packet. However, because two products are proxy-based, one is bridge-based, one is offline and some come with Web acceleration while others have built-in caching, we could not properly compare them and provide useful data. Ultimately, any enterprise running a WAF will also be running either built-in or third-party Web acceleration/caching servers.

All Network Computing product reviews are conducted by current or former IT professionals in our own Real-World Labs®, according to our own test criteria. Vendor involvement is limited to assistance in configuration and troubleshooting. Network Computing schedules reviews based solely on our editorial judgment of reader needs, and we conduct tests and publish results without vendor influence.

Imperva SecureSphere Web Application Firewall G4 Appliance with Software 4.2

From beginning to end, Imperva SecureSphere is our kind of WAF. Installation was a breeze. Unlike other vendors, Imperva recommends using a separate management appliance; our test setup included the G4 Appliance, a separate MX Management server and a Compliance Reporting Module for $45,000. The company also offers a single gateway product with an integrated management system that starts at $30,000. An organization could have as many as 15 gateways talking to one management appliance. The management appliance aggregates all suspected violations.

Unlike the other products we tested, SecureSphere runs as a bridge, so the setup required no changes to our DNS and only about 20 seconds of downtime on our test Web server--as long as it took to unplug the server from our switch and plug it into the SecureSphere gateway. While reverse proxies must terminate traffic for inspection, a bridge does not. Rather, the bridge holds all packets while it inspects each and then either forwards or rejects the request. This setup should add less latency than terminating traffic and rebuilding packets.

Once we plugged our Web server into the SecureSphere, it instantly began learning. While SecureSphere looks at all ports, its signatures and most management focuses on Port 80 traffic. The first thing we did was use SecureSphere's Application Defense Center to check for signature updates. We expected signature updates to be common to all WAFs we tested, but surprisingly only Imperva provided a way to update signatures. Over several weeks of testing, SecureSphere updated signatures at least three times. The latest number of signatures loaded was 2,167, across 10 signature dictionaries divided among Web server vulnerabilities, database server vulnerabilities and generic server vulnerabilities.

SecureSphere was the only appliance that not only allowed us the granularity of looking at signatures, but let us create our own signature dictionary. After clicking on "Show Signatures" under the "Worms and Critical Vulnerabilities for Web Server Groups" dictionary, we were presented with a large number of signatures plus useful information, including: what services the signature applied to; attack details, such as the complexity of the attack; what systems and OSs are affected--Linux, Unix, Windows; and references, including links to the Common Vulnerabilities and Exposures database and the Snort signature database. We also liked that each signature group had a set of default rules that identified the level of alert (informative, low, medium, high); what type of action to take if the signature is found (block); and what type of follow-up action to take. For example, during testing we had the SecureSphere send an e-mail if a SQL injection attack occurred; in other cases we had it do a short- or long-term block of the sender's IP address.

In addition to signatures, SecureSphere comes with plenty of built-in rules, broken down into protocol violations, profile violations, correlation and firewall rules. For each rule, the appliance comes pre-configured with what type of alert and or action to take. For example, we tested session hijacking, which was defined as a high alert under profile violation rules. We were given options as to how the rule is invoked--for example, which attributes must change in an active session to identify the traffic as session hijacking.

SecureSphere's learning mode is also outstanding. Without any configuration, the device started learning all the URLs that are accessed in our Web environment. Although all the other products required us to set up the virtual servers or Web sites we wanted them to protect or learn, SecureSphere helped identify all traffic coming into the box, which made it easier to identify Web profiles. Even when the product has "adapted," it still learns of any new requests that do not fall into a defined policy. For example, after several hours of being plugged in, SecureSphere identified more than 30 hosts (virtual servers) that were accessed from the Internet. We not only found a host that was being run without our knowledge, we also found improper URLs being accessed in our development environment due to absolute URLs accidentally left in code after it was moved to the live environment.

It's important to note that though learning mode is a feature that we think helps Web administrators, you should not rely solely on it to protect your environment. Quite a bit of tuning is required to understand the business logic of specific Web applications.

For each host, we were able to keep the appliance in learning mode or switch to protect mode with the click of a button. In addition, it was interesting to see all the URL parameters that were evoked as part of a URL. For each parameter, SecureSphere learns the type of characters commonly part of the parameter value. We were able to see that a specific parameter value allowed a minimum of 0 characters and a maximum of 7,666 characters, for example, and we learned the breakdown of allowable characters it saw as acceptable.

Finally, the alert console was descriptive and not overwhelming. SecureSphere was unique among devices reviewed in giving us the option to aggregate a specific type of alert, in contrast to Microsoft's Event Viewer, where you have to scroll though hundreds of the same event type. After running some vulnerability tests--specifically SQL injections and XSS (cross-site scripting)--we clearly saw the attacks in the alert center. For each attack we could see the specific violation. In our case, we saw the signature that was violated and the complete HTTP request, including all parameters, header information and cookies--invaluable information for determining if a blip is a one-time attack, a series of attacks, a vulnerability in the code, or something else altogether.

In case of a profile violation (signature match), we could jump directly from the alert console to the specific rule and make changes. However, if it was a protocol violation, we had to take note of what occurred, then find the appropriate rule to change.

F5 Networks Big-IP Application Security Module
F5 is well known for its Big-IP line of load-balancing, traffic-management and traffic-acceleration products. With this product, F5 adds Web application security to the suite. The Application Security Module is available both as a stand-alone appliance, powered by F5's TMOS, or as a software module that runs on the Big-IP system. For this review, F5 provided us with the Big-IP appliance, which can be extended by adding different modules.

As you might expect, given F5's extensive background in networking, the device could be deployed in a variety of configurations in conjunction with Big-IP. We did most initial configuration using Big-IP's management interface, rather than within the device, which provides access to Big-IP's powerful traffic-management capabilities. However, with great power comes great responsibility...or, in this case, a great user manual. We had to familiarize ourselves with traffic pools, virtual servers, application security classes and health monitors. It wasn't encouraging to find that each of these came with myriad configuration options--and we hadn't yet delved into iRules, Profiles, Nodes or a dozen other features.

Conceptually, to set up the firewall, we had to first create two "pools." A pool is a collection of devices that can be used for load balancing. The first pool contained both the WAF and our Web server. Big-IP load balances incoming traffic to this pool, while this module has a higher-priority setting and thus gets traffic directed to it. If this device were unavailable for some reason, traffic would be directed to the Web server, thus effectively providing a fail-open capability.

Once the firewall processed our traffic, based on its policy, it returned the request to Big-IP for load balancing to the second pool, which contains only our Web servers. This link between the Application Security Module (ASM) and the TrafficManager is created by setting up an application security class with the appropriate settings.

This module primarily employs a positive security model, though it also has a customizable list of negative regular expressions that function like attack signatures. Once the firewall was set up, it started monitoring traffic and, by virtue of its positive model, promptly blocked everything. We recommend disabling blocking until initial setup has been completed. As it monitors traffic, the device categorizes violations across categories, such as "illegal object type," "illegal parameter" or "illegal method." The administrative interface allowed us to click on these categories and see the details of each occurrence.

The Big-IP module monitors all the domains specifically entered; in contrast, SecureSphere automatically builds a picture for all Web sites on the server. The main difference between the two, however, is how they logically group and display data--the device's interface isn't as intuitive. While SecureSphere displays information grouped by Web page, Big-IP might show 10 "illegal parameter" violations. Selecting one shows individual incidents, rather than grouping them by Web pages, making it difficult to figure out which pages have been learned and which haven't. Say you feel the parameters for Page A have been learned correctly; you should be able to switch it to protect mode, while leaving other pages or sites in learning mode. For example, under illegal object types, we found .ASPX and .GIF extensions along with information on request length and object length. We could accept the currently learned configuration or adjust settings manually to determine which activities should be allowed. In this case, we could tweak the allowed request/response length--we have an .ASPX file that accepts uploads, and while the uploaded files are typically small, we want to manually allow a larger request length to allow for the occasional large upload. This can be cumbersome to begin with but does allow for really fine-grained control.

For those in a hurry to set up an initial profile, Big-IP provides a Web crawler that can be pointed to your Web site and turned loose on all the pages. This will ensure that it discovers all possible object types and lengths in order to build a good initial picture. You could then go in, accept values and switch to blocking mode. Is this better than sniffing out requests to build a profile of sites and/or URLs? It's not an either/or proposition. Sniffers help uncover the kinds of inputs the pages accept on a day-to-day basis, which a crawler would have no way of knowing, but might remain ignorant of rarely visited pages. A crawler, meanwhile, will touch on all pages in your site, assuming there's logical linking to every page.

One unique way in which F5's device provided control over the Web application, was by allowing us to define "flows," or allowed paths between pages. This enabled us to block people from manually entering URLs to navigate directly to an internal page. All this control came at a cost--the Big-IP required the greatest amount of manual intervention in setting up the profile. Fortunately, F5 did make the process fairly easy. One disappointment, as we checked off allowed object types and parameters, was that we weren't able to limit these to a certain URL, rather than across the entire site.

The management interfaces for both Big-IP and this module are Web-based and secured using SSL. While the overall interface didn't feel as intuitive as some of the other products, it was straightforward once we had familiarized ourselves with it. Whenever we weren't sure how best to proceed, we found the context-sensitive help, available on each page, genuinely useful--more than can be said for many products.

The module's forensic reporting interface doesn't group incidents by type, but provides a powerful filtering engine, which allowed us to filter events by violation, IP address, time, request type, string matches and more. We feel default grouping provides a better overview of activity than having to manually adjust the filter. Interestingly, as a default feature, F5's WAF provides a SupportID number to the client every time a page is blocked. This number can then be searched on the "Forensics" page to retrieve the exact event and see why it was blocked. If it was a false positive, we could click on the "accept" button to automatically modify the current policy and prevent the client from being blocked in the future--a useful and unique feature.

In comparison to Breach and Imperva, however, F5's device lacks detailed information on the attacks, explanations about their severity, or references to further information. This may not be an issue for a knowledgeable administrator, but we think it's a good idea to provide a little guidance on how severe a threat is and what a suitable response might be. While there is a wealth of information in the log files, reporting options were limited. We were able to export events as an XML file, but would like the ability to generate basic charts and perhaps export to Excel--not much to ask.

Overall though, we found F5's offering to be a solid product. Big-IP shops should certainly consider it, but it made a favorable enough impression for us to advocate deploying the standalone appliance on its own merits. Our as-tested price was higher than rivals', at $64,495, but included F5's Big-IP platform, with a built-in load balancer. Organizations looking for a stand-alone WAF should check out F5's newly released TrafficShield 4100, priced at $27,995. This stand-alone version was not available in time for our testing.

NetContinuum NC-1100-AF

NetContinuum came out of the world of network firewalls and can work in virtually any mode, including bridge, router, offline and reverse proxy, the mode we used during our tests. Although there are a lot of configuration options that can be set within the network portion of the firewall, we did not test those features for this review.

Configuration took longer than with either the Breach or Imperva offerings, and required some changes within DNS, to point our A records to the Security Manager Appliance. After the network was up and running, we went into the configuration tool and added a series of Web servers, on both Port 80 and Port 443. We could have done setup using NetContinuum's Web Application Wizard, and while the wizard certainly saved time, it wasn't overly impressive; for example, we could not apply specific rules within the wizard.

NetContinuum uses a positive security model and had far fewer out-of-the-box signatures than either Imperva or Breach. We found the standard injection, XSS and directory traversal, plus many signatures related to credit cards. Although the NetContinuum device lacked the depth of signatures of its rivals, it did make it easier than any other product to add custom signatures using its regular-expression GUI.

Because this appliance is not learning based, it required quite a bit of up-front configuration and in-depth knowledge of our Web applications. Most configuration was done using the Web ACL interface, where we could add and edit URL-, parameter- or header-based ACLs. For example, we added several URL ACLs to account for our virtual servers. For each URL ACL, we could specify which domain it was applicable to, whether it was in a passive or active mode, and which URL we wanted to protect (/admin). In addition, we could turn on DAP (Dynamic Application Profiling)--this capability is a differentiator for NetContinuum.

NetContinuum's Real-Time DAP and Enforcement works on a per-session basis, in real time. When activated, it learned our applications on-the-fly, allowing us to create exceptions to tighten or loosen learned behavior using a policy recommendation wizard. NetContinuum's Real-Time DAP runs passively or actively, either learning and logging or actively enforcing as it learns. The company warns that enabling DAP within a high-traffic site will add latency. However, it's important to note that this is different from pure learning-based WAFs: NetContinuum says it believes its strength is using a positive security model that denies all requests by default. DAP is the exception, and NetContinuum does not recommend using it in high-traffic environments or for long periods of time.

In a learning appliance, such as those from F5, Breach or Imperva, the device would learn URL lengths, content lengths and other information. However, with NetContinuum's product we would have to specify this information ahead of time to build a policy.

After setting up the URL ACL, we added several parameter-based URLs. Again, this was a painstaking process because we had to work with our developers to understand all the parameters in our Web environment and how they are being used. Within this interface we could block specific types of parameter-based attacks, such as XSS or SQL injections. However, we experienced many false positives during the early stages of setup, as we continuously tweaked these settings for our environment.

The best part of the NetContinuum NC-1100, as it relates to WAF, is its Web Firewall Logs Console. Here we were able to see detailed data on attacks taking place during our testing. The best feature was the exception-based GUI, which allowed us to apply exceptions based on specific URLs and rules with the click of a button. For example, one of our applications was denied because there was a tilde in the URL, which qualified as a path traversal attack. After reviewing the exception and clicking "apply," the appliance automatically created a URL ACL with the proper settings to allow this request in the future.

We were disappointed that NetContinuum logs did not group attacks--we ended up with a list of hundreds of possible attacks that we had to review individually. In addition, the NC-1100 did not have the level of log detail we found in Imperva's SecureSphere or F5's Big-IP. For example, attack explanations were brief at best, with no links or specific information to help educate Web administrators. Attack details were average, but did not show full header information--something we find useful. Reporting is also average, with simple graphs and charts showing basic information. Logs could only be exported to a CSV file; it would have been nice to have other export options available, or a built-in reporting tool.

Breach Security BreachGate WebDefend

Breach is unique in our test by not placing its appliance between the Web server and clients. Instead, the BreachGate sits alongside the server and monitors traffic through a hub or a mirrored port on a switch. This makes deployment very easy, requiring no changes to DNS or Web server settings. More significantly, it does not add another point of failure on the network.

The appliance uses three network ports, one each for management, traffic inspection and blocking...which brings us to one downside of the offline approach. Because clients connect directly to the server, offending connections are terminated by sending TCP reset packets to both--obviously not a completely reliable defense. The BreachGate needs a certain amount of time to analyze traffic and detect threats. In a low-latency setup, such as our lab environment, we were able to send an attack and receive the complete response from the server before the reset packet arrived. To minimize this danger, Breach recommends putting the monitoring port closer to the edge of your network and the blocking port closer to the Web server--this gives the appliance a bit of a head start by maximizing the time available to intercept an attack. In addition to TCP resets, BreachGate also functions with an existing infrastructure to provide for IP address/TCP session-based blocking through firewalls, and monitoring/reporting using SNMP.

Once physical installation was complete, getting the BreachGate up and running was straightforward. We added our test Web site to the Site Manager and selected a security policy, at which point the BreachGate began running in learning mode. As it monitored traffic, the appliance built a picture of our site, which it displayed in a tree view within the Site Manager. Selecting a page showed the parameters used. Double clicking on a parameter allowed us to define it in great detail: where it appears on the page (query, content or both), minimum and maximum length, data type (e-mail, date, positive integer, list), and whether it's hidden. Overall, the interface was quite intuitive and easy to use.

When the BreachGate determines that it has a large enough sample size for a given page or object, we had the option to allow it to automatically switch from learning mode into protect mode. Note that the move to protect mode is for that particular page only. It will continue to learn the pages that haven't yet been sampled.

The Policy Manager lists BreachMarks and security rules. BreachMarks are regular expressions used to flag certain types of data, like credit-card or Social Security numbers. They serve two purposes. First, an information leakage event is recorded each time data that matches a BreachMark is identified in a response. This allows you to log all instances where sensitive data has been displayed on your Web application. BreachMarks can also be used to mask sensitive data in recorded events, to prevent it from being visible in logs.

In addition to BreachMarks, the Policy Manager lists a large number of security rules, which are analogous to signatures. These are grouped logically into folders and sub-folders, and color coded based on severity, and can be activated and deactivated either singly or as a group. Responses to security events, however, have to be tuned individually. Selecting a rule showed detailed information on the threat, along with links to security Web sites where applicable, and allowed us to select one or more possible responses (log, alert, reset, block) for each possible attack outcome (success, fail, information leakage, attempt). Again, we found this function quite intuitive.

Grouping BreachMarks and Signatures forms a policy. We were able to create several policies by activating sets of rules and selecting responses, which allowed us to employ different policies for different Web sites. However, we weren't able to add custom security rules to the policies, or tweak existing rules to, for example, change the definition of a SQL injection attack.

The last major component of the BreachGate WebDefend console is the Event Viewer, which allowed us to view a history of events for a given site. A list of events can be viewed either individually or grouped by type. We could also apply several filters, including date, severity, IP address and URL--useful for tracking a particular attacker or incident.

Selecting an event provides a detailed view, listing which security rule was applied, general information on the threat with references to security sites, and complete details of the HTTP request. A feature unique to the BreachGate enabled us to review the response sent by the server, both as raw output and in a browser view. In a worst-case scenario, this could be especially useful in determining what exactly was compromised and how much information the attacker obtained. With new laws requiring disclosure of any security breach to affected customers, this feature can help you determine whether you need to e-mail ten people or take out a full page ad in the newspaper. The data is stored on the appliance, as part of its event tracking. The response is associated with a particular event and will be deleted once the event is cleared.

The BreachGate is a great WAF, but that's all it is...it doesn't have any of the network firewall, caching or load-balancing components found in F5's and NetContinuum's products. In addition, troubleshooting and reporting are average at best. We ran into bugs while trying to export reports, and help files are basic and non-interactive. Still, we believe that within the next two to three releases, Breach has the potential to be a major competitor in the market.

The BreachGate WAF is reasonably priced, starting at $24,995. Breach also offers larger systems that can handle higher traffic volumes and include options such as SSL acceleration.

R E V I E W

Web Application Firewalls

Welcome to NETWORK COMPUTING's Interactive Report Card, v2. To launch it, click on the Interactive Report Card ® icon above. The program components take a few moments to load.

Once launched, enter your own product feature weights and click the Recalc button. The Interactive Report Card ® will re-sort (and re-grade!) the products based on the new category weights you entered.