UPS Taps 'Continuous Analytics' for IT Systems Reporting

Continuous intelligence, continuous analytics, and operational business intelligence -- the concept of real-time BI goes by several different names and is being pursued within several different technology camps. But what you often hear about is neither real nor timely, with vendor promises in place of real customer case examples and fast batch processing passed off as low-latency information. That's not the case with the very real "continuous intelligence" application implemented at UPS. The shipping giant completed an event-stream monitoring application last summer that put its legacy, rear-view-mirror reporting approach to shame.

UPS started considering real-time intelligence options as planning got underway to replace 12 legacy IBM iSeries (AS/400) servers with 20 commodity Intel servers running Linux. Split between the shipper's mirrored data centers in Mahwah, NJ, and Atlanta, the legacy servers powered an application that handles tracking requests from the UPS.com front end as well as shipping requests from the WorldShip application on customer PCs. The application load balances and routes those requests to the shipper's mainframe-based, back-end applications, handling as many as 50 million transactions a day. A key downside of the legacy app -- apart from the cost of maintaining it on proprietary hardware -- was the lack of visibility and timely reporting. "We were collecting log data from each of the 12 servers overnight, consolidating that onto a single reporting server and producing a series of statistical reports on transaction attempts, successes and failures by server," explains Jim Saddel, a systems manager at UPS. "We had nothing in the way of real-time reporting. To see what was going on in the moment, support people had to log on to each of the 12 servers and run SQL queries."

As part of the larger migration project, UPS wanted real-time visibility into system performance while also automating monitoring and alerting activities formerly handled by a separate IT department. After consulting with the UPS Enterprise Architecture team, Saddel says he was encouraged to consider continuous analysis technology from Truviso. Not only was the technology already in use and accepted as a standard within the company, but the UPS Strategic Enterprise Fund -- the venture capital arm of the shipper -- had invested money in Truviso. A pilot test was arranged in late 2007.

"Truviso took a sample of our log data, and within about a week they developed a pilot dashboard with all of the [reporting] elements we told them we wanted," Saddel says. "When we did a buy-versus-build analysis, the bottom line was that for what it would cost to develop an in-house solution to get once-a-day reporting, we could get so much more."

UPS also looked at Splunk, an indexing and search engine for log data that is also in use within UPS. Saddel says Truviso was selected for its real-time data summarization capabilities as well as its proven high-volume processing capacity. Truviso describes its technology as "continuous query" software that applies the speed and processing capacity of complex event processing (CEP) and business activity monitoring (BAM) technologies to the BI-oriented job of exception, on-demand or real-time reporting.

"The BI industry doesn't fully realize how deeply and quickly CEP is beginning to permeate reporting, dashboarding, analytics and other decision-support apps," Forrester Analyst James Kobielus recently wrote in this blog post. "Clearly, changed data capture and low-latency, end-to-end propagation is a middleware area where CEP and BI overlap with best practices for near-real-time, 'active' data warehousing."

At UPS, BI and data warehousing technologies weren't in the picture, but the objectives of the deployment should sound familiar to anyone hoping for real-time BI and low-latency data warehousing. Once Truviso's technology was rolled out in July 2008, reports that were previously delivered overnight were available instantly from real-time dashboards and up-to-the-minute reports.

"Now when we're doing an application installation, a software upgrade or a traffic fail-over, we can watch our dashboards and see right away whether we're seeing what we expect to see," Saddel says. "The applications support staff can see whether we're load balanced within and across the data centers and whether we're operating within expected tolerances."

Upgrades of the deployment UPS implemented in April 2009 added real-time reporting of server-level operating information, connection details, system performance "high-water marks" and information used in capacity planning. UPS has also implemented e-mail- and phone-text alerts for "status yellow" conditions that help IT managers take proactive steps to head off out-of-tolerance performance conditions.

When problems arose with the old system and IT peers called Saddel's group to ask what was wrong, more often than not, all they could tell them was "we don't have a dashboard, so we'll tell you tomorrow what we see," Saddel says. "Now we can look at the dashboard and see right away whether it's an across-the-board problem or an isolated problem on a specific server."

For UPS, as for any technology-dependent company, IT visibility translates directly into business execution, so real-time reporting is far more significant, in this case, than a keep-the-lights-on IT application.