Third installment of our Ajax Rolling Review shows not all scanners are created equal
Web application scanners in this Rolling Review must not only find traditional vulnerabilities, like XSS and SQL injection flaws, but also handle Ajax applications, in which part of the app is running locally in the browser.
Complex Ajax applications represent a new twist for these products, and we don't recommend purchasing a scanner that isn't able to handle Web 2.0 environments, given that so much future development is moving in that direction. And Web application scanners should be just one element in a comprehensive, layered program--educating developers and integrating security reviews into the development life cycle are just as crucial.
N-Stalker's scanner failed to deliver on basic Web application security detection, let alone finding Ajax flaws. It does have the potential to be a useful scanner for known vulnerabilities once some quirks and bugs are cleaned up, but it simply can't compare with the first two products in this Rolling Review.
The range of products calling themselves "security scanners" is so broad that the designation is flirting with irrelevance. You have your vulnerability assessment software, which uses large databases of known vulnerabilities. Then there are penetration-testing applications that focus on fewer vulnerabilities but include the ability to exploit flaws instead of just identify them. More relevant to this Rolling Review are Web application scanners, which attempt to uncover problems in newly developed software--before they get exploited.
As an added twist in this review, we've focused our testing on Ajax applications. We've already evaluated Hewlett-Packard's WebInspect (formerly from SPI Dynamics) and Cenzic's Hailstorm. Both are Web application vulnerability scanners aimed primarily at crawling new Web apps looking for exploitable flaws. Sure, they're able to detect some common misconfigurations within Web servers and languages, even pick up a few stock bugs in known programs. But that's not their primary focus.
LAGGING THE FIELD
Unfortunately, the newest entry in this Rolling Review, N-Stalker's Web Application Security Scanner 2006 Enterprise Edition (try saying that five times fast), doesn't measure up to the previously tested scanners, despite its hefty built-in database of vulnerabilities in known Web servers and Web apps.
With three iterations of the product--the QA Edition; the Infra Edition, for infrastructure scanning; and the Enterprise Edition, which includes the QA and Infra versions as well as audit and penetration-test capabilities--N-Stalker has a great conceptual approach. On paper, it looked like an ideal fit for this review--we're looking for products that take into account the different potential use cases for application scanners, and on the face of it, N-Stalker's three-pronged approach is perfect.
Unfortunately, while the QA and Infra offerings may be somewhat useful thanks to their large built-in vulnerability databases, the audit and penetration-test modes are plagued not only by poor detection capabilities for new vulnerabilities but also a severe lack of tools to aid in advanced manual penetration.
In our evaluation, N-Stalker's scanner failed to find a number of vulnerabilities that all of the other products were able to identify. Additionally, the engine was too easily caught in unintentional scanning loops on one site that generated recursive links. Without recognizing the subsequent URLs as having repeated identical variables, the product was tripped up.
From a usability standpoint, N-Stalker's scanner not only fails to hit the bar set by WebInspect, it doesn't even compare well with the weaker interface found in Cenzic Hailstorm. Adding credentials for an application was a trivial matter with both WebInspect and Hailstorm, for example, but not only did N-Stalker fail to include any kind of automated login detection, even using the manual process was tedious, requiring at least twice the number of mouse clicks and keystrokes as the rival products did.
Numerous other usability flaws and outright bugs abound: Multiple application windows that randomly failed to display in the Windows taskbar. Buttons silently failing to work. Having to guess the next necessary step, nonresizable windows hiding necessary data, and more. N-Stalker says it is addressing at least some of these usability issues in its 2007 Edition release, due in October.
We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
2017 State of IT ReportIn today's technology-driven world, "innovation" has become a basic expectation. IT leaders are tasked with making technical magic, improving customer experience, and boosting the bottom line -- yet often without any increase to the IT budget. How are organizations striking the balance between new initiatives and cost control? Download our report to learn about the biggest challenges and how savvy IT executives are overcoming them.
Infographic: The State of DevOps in 2017Is DevOps helping organizations reduce costs and time-to-market for software releases? What's getting in the way of DevOps adoption? Find out in this InformationWeek and Interop ITX infographic on the state of DevOps in 2017.