The developers added a stored procedure--a specialized SQL query routine that's used so often that it's stored inside the database--but it wasn't listed correctly in the DB2 index. Consequently, the following night's batch processing slowed as calls for the stored procedure went unanswered.
The bank's customers, such as institutional investors and asset managers, rely on reporting from that application and could have come to work the next day and found those reports missing, says Jason Bonds, senior technical architect at Northern Trust's operations and worldwide technology group. "We might have started getting calls to the help desk," he says.
Instead, the slowdown was noticed by Wily Technology's Introscope 5.0 application-monitoring system, installed on 16 BEA Systems Inc. WebLogic application servers at the bank. The level of output from the application querying the DB2 system had dropped below an acceptable threshold, and night operators were notified by Introscope of that development. They in turn awoke a database base administrator who was on call and found the error and re-indexed the stored procedure, which started the system humming again. Trouble averted.
It's an example of why Northern Trust, which already monitors its systems with BMC Software Inc.'s Patrol system-management monitor and Mercury Interactive Corp.'s Topaz online application monitor, also uses Wily's Introscope. Patrol would tell the night operators that the DB2 database server and the application server were running, but it wouldn't necessarily say a slowdown was occurring in an application.
Topaz monitors an online application from a user's response time point of view, but it doesn't look inside batch processes, and it wouldn't be able to sound an alert that a report expected by customers in the morning wasn't likely to be there.
"We need all three," Bonds says. He adds, however, that "it was Wily that warned us of a slowdown."
Wily's Introscope was recently upgraded to version 5.0, and Bonds says the reach of its monitoring capabilities is much improved.
One thing that saves Northern Trust time and money is Introscope's ability "to probe the Java Virtual Machine," or Java runtime environment at distributed computing machines, rather than monitoring only the host level. Monitoring a distributed computing JVM yields statistics on how well a SQL query is running for a remote site without forcing a change in the core database driver to produce the needed statistics. A driver change costs Northern up to $3 million as it's tested extensively against the bank's infrastructure systems to avoid unintended incompatibilities and failures, Bonds says.
Introscope now works with the open-source JBoss application server as well as BEA's WebLogic, IBM's WebSphere, Sun Microsystems' Java Enterprise System, and Oracle Application Server.
Wily recently unveiled seven PowerPacks that extend Introscope's monitoring to the specific adapters and connectors of the IBM WebSphere Business Integration suite. Introscope can now monitor the performance of the WebSphere Business Integration adapters for mySAP.com, Oracle, PeopleSoft, and Siebel Systems enterprise applications. The PowerPacks also cover JDBC database connectors, JText text fields in a Java user interface, and HTTP communications. The PowerPacks start at $500 each per CPU. Introscope starts at $6,250 per CPU.