September 25, 2000

Printer ready

Web-Site Monitoring Derails Problems

continued...page 2 of 2

More on Web-site performance:

sidebar: To Optimize Performance, Look At The Last Half Mile InternetWeek: Anticipate Problems, Map Changes (9/4/00) InternetWeek: E-Commerce--Stay Afloat Without The Bloat (8/14/00) VARBusiness: Optimize Your Client's Site (8/7/00)

TechEncyclopedia:

Need a tech term defined?

Send Us Your Feedback

"We formalized monitoring consulting services so that companies can leverage our expertise to not only collect ongoing performance and transaction accuracy data, but to analyze and interpret that data to improve performance," Cohen says. In addition, Segue has completed a certified interface with BMC Software Inc.'s Patrol, combining its user perspective with BMC's infrastructure view.

William Johnson, CIO of online retailer Lucy.com, a Portland, Ore., seller of women's sportswear, approached the challenge of putting monitoring practices to work on several levels. "Performance monitoring is essential to our daily business," Johnson says. "We're a nationwide chain with one store, and the store must never close."

As a result, the IT team at Lucy.com watches site performance from multiple vantage points, using multiple tools from multiple vendors. "First, we needed tools to determine the health of the system infrastructure itself," Johnson says. "On another level, we needed to collect information about the user experience. We needed to be confident that our site was operating correctly and that response times and throughputs were meeting the acceptable levels defined by our load tests and ongoing transaction benchmark statistics. On a third level, we needed tools to characterize where our customers come from, how often they visit, what they actually do during their visit, and where they go when they leave. This third level of site monitoring is useful for our marketing initiatives and ongoing improvements to site design."

On the infrastructure level and because of its standardized use of Microsoft technologies, Lucy.com is implementing tools from NetIQ Corp., which will give the company a high level of detail about low-level system-diagnostic information, such as CPU utilization, memory, disk, and buffer cache. The current process requires the IS department to weed through server logs, but as the company grows and expands its infrastructure, this process becomes very labor-intensive. Implementing NetIQ will aid in systemizing and simplifying the process.

To gain a customer perspective, Empirix's e-Monitor tool is used in conjunction with Keynote Systems Inc.'s performance-monitoring service. "The Keynote service is very useful for collecting a customer perspective from outside of our firewall from multiple geographic locations and providing us with comparison data that tells us how our site stacks up against our competitors," says Chris Conger, a quality-assurance engineer for Lucy.com. "But e-Monitor is really the central tool for us because it gives us more flexibility to know what's happening on our site.

E-Monitor reports on what our users are seeing, but because we have control of it in-house, we're able to gain enough detail to conclude whether or not it's our problem." If that's the case, then additional tests are run and the operations group is pulled in to drill down and do more system analyses.

E-Monitor is an extension of Lucy .com's automated testing practices. Scripts developed to simulate multiple types of transactions for the test environment are used in the live environment to make sure key transactions are functioning according to business requirements. E-Monitor measures a handful of key transactions every two minutes for Lucy .com, and triggers alerts to performance exceptions, such as a page that doesn't load or is too slow, missing images, broken links, or transaction errors. E-Monitor is also used to monitor performance during load tests, and Lucy.com uses the tool to take daily measurements from outside its environment via an Internet service provider to continually update the performance benchmarks of its most frequent and critical transactions. This ensures that the company's performance benchmarks aren't static.

"The combination of the e-Monitor tool and the Keynote service is a good indicator or warning that problems exist," Johnson says. "We plan to expand our use of e-Monitor by using its VBScipting capabilities. With our own programming, we plan to pull detailed server log data directly into e-Monitor reports to help make the process of drilling down and solving problems even more efficient."

KeyCorp in Cleveland has taken advantage of another way to capture performance metrics in the live environment. This method requires going directly to the source and collecting transaction response time metrics from the real users that visit its site. Gaining user perspective on Web-site performance in this way is possible using Candle Corp.'s eBusinessAssurance system.

After testing site performance with tools from Segue, KeyCorp turned to Candle to manage its Web-site performance from the perspective of its Internet banking customers. Ranked as the 11th-largest bank-based financial-services company in the United States by American Banker, a daily financial newspaper, KeyCorp began offering Internet banking to its customers in late 1997. "We wanted to measure the actual user experience of our customers to ensure we routinely meet our service level agreements," says David Williams, project manager of technology services for KeyCorp.

KeyCorp collects performance metrics from about half of its 30,000 user sign-ons each day by using eBusinessAssurance to place a Java applet on the outbound HTML page requested by a customer. When the applet arrives at the customer's PC, it executes and collects time-based measurements. Upon return to the KeyCorp Web server, the applet is stripped of its pertinent statistics, which get pushed into a Web log. Web-log statistics are uploaded to Candle's eBusinessAssurance Network, or data warehouse, hourly for post-processing. Candle's subscription-based reporting service handles all analysis and interpretation.

Williams can access KeyCorp's performance metrics via any standard Web browser. The information is provided in report formats and helps him make decisions about the level of service his Web site is providing. Performance metrics at KeyCorp are generally reviewed on a monthly basis. "We do have the option to analyze our data in-house, but instead choose to outsource that task to Candle, freeing us from the requirement of having the dedicated data warehousing resources to accomplish the task," Williams says.

This method of monitoring gives KeyCorp measurements that are specific to its customers. The challenge may lie with customers who have opted not to accept Java applets through their Web browser. Measurements for these customers can't be collected. Also, for customers who don't have a Java Virtual Machine loaded on their sites, some response times may be affected by that load time. However, given the high volume of measurements that KeyCorp collects, the law of averages likely works in its favor.

return to page 1

Illustration by Marlena Zuber
Photo of Johnson by Shane Young

Back to This Week's Issue
Send Us Your Feedback
Top of the Page

---