Rolling Review: WatchFire Blazes Past Competition

Not only is AppScan the most mature Web application vulnerability scanner on the market, developed in 2000 as a companion to Sanctum's AppShield Web application firewall, but now IBM owns it as a result of its Watchfire acquisition.

We weren't sure AppScan's experience would be enough: The Ajax apps we've been feeding our scanners have proved troublesome, even for long-established products. However, AppScan impressed us with its ease of use and advanced functionality and reliability. It was the most successful so far at traversing our Ajax apps.

There were some tense moments though. When we scanned a sample Ajax app, AppScan was unable to automatically parse the JavaScript and enumerate the entire application. When Watchfire investigated, however, it said it had no trouble with the application. What gives?

The problem was traced to a rogue Microsoft XML library on our scanner that wasn't properly registering. Until the exact cause was tracked down, Watchfire added a temporary fix to the AppScan installer to ensure that the library worked properly. Customers needing JavaScript testing functionality should be sure they're running the AppScan Update tool to get the fix for this bug.

Once that issue was resolved, AppScan performed as promised, becoming the first Web application scanner we tested to properly identify Ajax functionality and navigate those sections of the app other scanners failed to automatically crawl.

AppScan encapsulated the best features of all the products in this review with few of the faults. It's cleanly designed, and the interface was easy to use, much as with Hewlett-Packard's WebInspect, but without its reliability questions.

For advanced users, AppScan's built-in utilities are nearly on par with the rich suite of tools integrated into WebInspect. Additionally, since version 7.5, AppScan has taken a cue from the Firefox browser, letting users develop extensions that can integrate into the product. These add-ons reflect the growing popularity of open source products and communities. One sample extension is a complete development environment in itself, integrating the popular open source scripting language Python with AppScan's core engine.

Much of the value in a scanner stems from how seamlessly it can be incorporated into your workflow to provide meaningful and actionable data throughout the development process. Exposing the product via extensions is a great way to let customers use AppScan in a way that best fits their particular needs or environment. Take the Pyscan module. An organization might implement custom scans of different branches of an application under development by automatically scripting both scanning and reporting as code is forked for reuse or checked into a source-code repository. Having the simplicity and popularity of Python tied to the scanning engine that makes AppScan tick is a powerful combination. Creative types will discover a variety of potential uses.

All this isn't to say that AppScan is flawless. Scanning an Ajax Web mail app was problematic until someone in the AppScan support group figured out that the Ajax application was detecting the User-Agent changing between the initial recorded log-in session and the rest of the requests, dumping the scanner back to the log-in screen. Once the cause was identified, it was a simple tweak to change the scanner's User-Agent to be consistent with the login script.

We were pleased by Watchfire's responsiveness. We have no reason to think the company won't be just as attentive to enterprise IT groups. Bottom line: AppScan isn't perfect, but it's more reliable, functional, and extensible and easier to use than any other product we've tested so far.