InformationWeek

SM-Web 2.1 allows detailed configuration of service-level agreement performance thresholds and alert notifications. A unique feature of this service is the escalating and de-escalating reporting lists that alert different groups of people for different performance variances. Each group is configured with a percentage of the original agreed-upon performance threshold; once the threshold is violated, the group is notified. For example, with a configured service-level performance threshold of 10 seconds for a page to load, first-level network and Web administrators can be alerted after a 10% limit (11 seconds) is reached. A second, third, or fourth group, comprising mid-and upper management, for instance, can be notified when higher limits are reached. Each group is configured with a percentage of the original service-level performance threshold that, once surpassed, will alert that particular group. Notifications can also be sent to differently specified groups when performance improves.

With its large number of configurable and customizable components and excellent graphing, Keynote sits near the top of our list. Keynote's services range from simple Web-site monitoring to end-to-end consulting and auditing services. Keynote provides varying levels of help that fit nearly any budget. Keynote's signature feature is the numerous groups of agents it uses to collect data. Keynote's agents can be chosen from 50 metropolitan areas to accurately emulate the type of traffic a customer expects. Dial-up and broadband users can be emulated to provide specific measurements of a Web site. Pricing ranges from $249 to $1,495 per month per URL, depending on the number of agents used in collecting data. Other offerings, such as consulting services and specialized agents, are extra.

MyKeynote's main page displays the monitored URLs and configurable indexes in graphs that depict the URLs' response times for the past 24 hours. The graph has hot spots at each data point to allow quick identification of performance problems and details of that specific data point. We found the extensive configuration options and the customizable user-note sections helpful for quickly identifying problems or suspicious performance levels.

Keynote's graphing features are richer and more refined than those of its competitors. For instance, only Keynote lets you click on the graph and zoom in. Using this drill-down feature, we zoomed into a section of a graph that showed slow response time for transactions. Once we isolated the slow portion of the graph, we could select the agent breakdown graph and see which of the Keynote agents had a problem. Depending on the agent group selected, the user can switch to various graphs to get more detail. Like Service Metrics, Keynote provides a time-line feature called "Diagnostics" that, although tabular in presentation, is equally effective at revealing Web-site performance data.

Keynote also provides feature-rich alert notification via E-mail. Configurable thresholds are present, although there's only a single group for all alerts, instead of Service Metrics' multiple group configuration. Alarms can be turned on and off for each URL and can be configured with URL-specific thresholds and tolerances.

Although it's not an identical feature-for-feature competitor to the others we tested, Evity offers some unique Web-site performance-monitoring capabilities. Primarily transaction based, it should be pitted against the transaction components of Keynote's, Service Metrics', and WebPartner's services. However, at the basic level, SiteAngel 2000 performs the same service and provides information equally valuable as that furnished by the other nontransaction-based services.

SiteAngel 2000's ability to monitor Web sites through a specific click path may prove more valuable in particular circumstances. SiteAngel 2000 rings in at $12,000 per year per critical path for the service we requested. That's comparable to Service Metrics' and Keynote's price points.

SiteAngel 2000 uses the convention of individual "angels," or agents, that collect performance data from a Web site based on a specific click path. Each agent is configured to repeat a learned behavior and collect statistical information. What makes SiteAngel 2000 unique is its ability to configure each step in the click path with performance data and attributes that must be satisfied by the Web site. This includes items or text directly from the Web page being loaded, as well as performance metrics regarding retrieval and transaction. Using three agents, we configured SiteAngel 2000 to load the base pages of our three test URLs. In this configuration, SiteAngel 2000 accomplished many of the same tasks as the other services.

The main SiteAngel 2000 page neatly lists all aspects of the customer's account and specifies agents available through their subscription. Each agent is listed with time stamps of the last and next planned runs, as well as a status indicator that tells whether and how frequently the agent is running. Although that frequency ranges from every half-hour to once every four hours, SiteAngel 2000 can't monitor the site as frequently as Keynote and Service Metrics.

Agent click paths are configured by surfing the site to be monitored. Client traffic is directed through a proxy server on Evity's site and, once the click paths are completed, the user can configure content checks and performance thresholds for each step. The entire setup process takes time--approximately 15 minutes for our site--but creates a click path that closely reflects the type of client session for which you want to gather data. This type of monitoring is especially useful for Web sites that require rigid content and click paths.

Reporting and analysis features are less useful than those of Keynote and Service Metrics. Evity's graphs made it hard to grasp our site's overall performance. Data is charted according to performance goals and a generic customer-satisfaction value calculated by comparing measured performance with performance goals and thresholds. Reports are text-based and brief, relaying only composite data for the report period. We appreciated the problem-resolution notification when a problem with our site ceased.