Right in Time - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Software // Information Management

Right in Time

To survive today's challenges, businesses must take a long look at their data warehousing investment - and break traditional barriers to delivering strategic and tactical information faster.

The New Economy, like its predecessors, proved unable to overcome business cycles. However, there's no question that economic developments in recent years have dramatically changed the global business landscape. Deregulation and globalization have removed many business barriers. To survive competitively, many organizations have had to reinvent themselves.

Transactional applications deployed have become silos of information, and do not meet the business intelligence demands of analysts and users functioning under different conditions. Collating, integrating, and reporting on data spread across heterogeneous platforms and applications remain major stumbling blocks in information access and delivery.

To address this challenge, many organizations have developed an enterprise data warehouse (EDW). The EDW architecture integrates data spread across transactional systems into a central repository, against which users may perform business analytics. The EDW is then loaded periodically with new data from operational systems. Reports, mostly strategic in nature, came to business users after an acceptable time lag. Overall, the EDW approach has served business needs well.

However, the recent economic slowdown marked by business contraction and deflation has forced companies to work hard at optimizing business processes, improve efficiency, and do everything possible to reduce costs. Most have an urgent need to maximize return on past capital investments, leveraging the most out of infrastructure systems already in place. What is the best way for companies to gain ROI from their data warehouse investments?

Latency Be Gone

First-generation data warehouses focused on reporting; the second generation brought online analytic processing (OLAP) and data mining. Sophisticated, multidimensional OLAP tools came to the market, which vastly eased complex data analysis through features for drilling down and pivoting on data, building and applying models for predictive analysis, and using the Web to publish reports.

The new generation, in response to current economics and business requirements, is focused on doing what is necessary to speed information cycle time, drive out information latency, and thereby enable users at all levels to be more effective. Information on demand is what businesses need. A real-time enterprise has to have up-to-date information for decision-making and for optimizing critical business processes.

The requirement to deliver information on demand for active decision support is fueling the need for real-time (or perhaps more appropriately called "right time") data warehouses, business activity monitoring (BAM), and alerting. These advances enable organizations to more efficiently execute business strategy by making optimal use of data resources. The real-time data warehouse (RTDW) focuses on cutting down latency in the decision-making process by empowering a wide spectrum of users. Information overload is reduced through monitoring business performance in the background and only reporting back exceptions to norms. RTDW also positions information better for use by downstream applications, such as customer relationship management.

How does the architecture of a RTDW differ from that of a traditional data warehouse? I should first say that an RTDW does not need to be built in complete isolation; organizations with traditional data warehouses in production can extend them with help from suitable tools and techniques. Most organizations will evolve their data warehouses toward hybrid architectures, such that shown in Figure 1. How much and how well you can extend the traditional system depends on the scope of "information on demand" requirements and the level of latency that your organization deems acceptable. If careful considerations are not made about this scope, the costs of moving to an RTDW can rise.

Figure 1 Real-time data warehouse — logical architecture.

Here are some key differences between a RTDW and a traditional data warehouse:

Tactical vs. strategic. A traditional data warehouse is inherently passive in nature; it only supports strategy formulation by selected users. It enables long-term planning and strategizing by looking at the historical trends. A RTDW focuses on improving tactical decision support and the execution of the strategy.

Real time vs. batch. As the name suggests, a RTDW delivers information on demand. It integrates the latest data from transactional systems with historical or contextual data to provide the most up-to-date view of the business. The queries work on lower data volumes, analyzing trends for isolated business cases. By contrast, a traditional data warehouse is batch-oriented; it is used for offline analysis. The most demanding queries in a traditional warehouse work on very large volumes of data to hunt for hidden patterns often across the entire spectrum of a business dimension.

Integrated vs. isolated. A major intention of a RTDW is to integrate data warehousing with business processes and the application systems supporting them. Analytical CRM applications offer a classic example. A traditional data warehouse would perform customer segmentation, scoring and life-time value computations as isolated batch processes — and this may be good enough for most segment-based marketing. However, businesses that desire highly interactive, personalized customer conversations need to have integrated, real-time analytics to support their efforts to establish positive customer interaction.

Guaranteed vs. best effort. A traditional data warehouse generally operates offline and delivers only on a best-effort basis. It guarantees neither availability nor performance. This is acceptable, as some delay in delivery of strategic information does not significantly erode the value of the information. In contrast, a RTDW must support active, ongoing decision-making and therefore needs to guarantee both availability and performance. If the active data warehouse consistently fails to deliver, the negative impact on the business could be significant.

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
1 of 3
Comment  | 
Print  | 
More Insights
CIOs Face Decisions on Remote Work for Post-Pandemic Future
Joao-Pierre S. Ruth, Senior Writer,  2/19/2021
11 Ways DevOps Is Evolving
Lisa Morgan, Freelance Writer,  2/18/2021
CRM Trends 2021: How the Pandemic Altered Customer Behavior Forever
Jessica Davis, Senior Editor, Enterprise Apps,  2/18/2021
White Papers
Register for InformationWeek Newsletters
Current Issue
2021 Top Enterprise IT Trends
We've identified the key trends that are poised to impact the IT landscape in 2021. Find out why they're important and how they will affect you.
Flash Poll