Not Your Father's ODS - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Software // Information Management

Not Your Father's ODS

A conventional operational data store (ODS) always hits a wall when the goal is producing actionable information. Here's how a carefully reengineered ODS could solve a wider range of enterprise information requirements.

First the good news: More than ever, business and technical users have access to a broad supply of information coming from across the enterprise. Organizations are utilizing and leveraging unique knowledge assets, thanks to growing deployment of enterprise information portals (EIPs), which provide a single gateway to heterogeneous data sources and applications. EIPs also serve as a platform for personalization and security tailored to the user's functional business role in the organization.

EIPs have broken ground for the development of performance management dashboards, scorecards, alerts, business process notifications, business intelligence (BI) reporting, e-mail and other mission-critical technologies for information-driven organizations. This reliance is driving business and technical users to collaborate to establish enterprise information management and architecture initiatives, such as master data management, data warehousing, metadata management, data quality management and data distribution services.

But now the bad — or at least challenging — news: The difficulties in supporting enterprise information management today and into the future are growing in size and scope, outpacing many organizations' ability to keep up. Users need information 24 X 7; especially for global organizations, data integration must occur around the clock. Maintenance and upgrade windows are shrinking. Creating more redundancy (typically by using failover systems) is an obvious option, but the cost and complexity associated with this approach lessens the appeal. IT management is in a quandary.

In this article, I'll propose a solution that builds on a key investment already in place at many organizations: the operational data store (ODS). Experience with ODS implementations is growing at a propitious time — a time when IT desperately needs to think outside the box about where to take its enterprise information strategy. After considering where things stand now, I'll discuss how the ODS may be repositioned to serve broader enterprise information requirements.

Necessity, Mother of Invention

A traditional ODS most often comes into being as a component of an overall data warehouse environment. The motivation is generally to support faster operational decision making with more focused, subject-oriented and time-sensitive data than the historical snapshots found in a traditional data warehouse. An insurance company might deploy an ODS to speed up its claims processes; an e-commerce retailer might use an ODS to interpret, in real time, what customers are doing on its Web site.

An ODS integrates detailed, operational data from different sources into a single store, without the history, summarization and extensive transformation typical of a data warehouse. It's set up to serve many standard, simpler queries against minimally transformed (if not atomic) data rather than complex, ad hoc queries against often extensively transformed and aggregated data. With its emphasis on frequently updated information, an ODS is usually smaller than a data warehouse, which might contain many years — and many terabytes — of data. Where a data warehouse might store an entire history of customer contacts, for example, the ODS would store just the names needed currently by marketing or accounting business functions.

Some companies try to combine ODS and data warehouse activity into one system. Business requirements, the number of operational systems and complexity of data transformation and integration processes, however, have led organizations to separate the structures schematically and often physically. Some ODS deployments serve nearly automated tasks, such as determining whether a credit card transaction is fraudulent or whether a customer meets profitability or other criteria for loans. An ODS frees both the transaction-oriented operational systems and the data warehouse from having to balance information delivery processing loads and prioritize operational, tactical and strategic queries.

An ODS might be the perfect antidote for the pressing enterprise information availability challenges I discussed at the beginning of the article. However, a traditional ODS still lives downstream from the operational data, which means that goals of producing immediately "actionable" information will always hit an architectural wall. But what if we could reposition the ODS to improve overall information availability?

Center of the Storm

IT managers should consider positioning their ODSs to be at the center of their enterprise information environment, as depicted in the diagram below. This would involve thinking of the ODS as more than just an adjunct to BI and data warehousing systems. A reengineered and redeployed ODS, in conjunction with an EIP, would help organizations address the greater availability, scalability, performance and security issues that challenge managers of heterogeneous business information systems.

How do you make the transition? Let's first consider more closely what we're dealing with. Numerous applications, involving many business areas, make up a typical enterprise information environment. Each one contains its own unique set of data, organized to fulfill the purposes of the business function the application supports. Recruiting systems create requisitions; payroll systems create pay runs; and billing systems create bills. In addition to the unique data, each application has a core set of information that's essentially common to most of the operational systems; these data objects include customers, employees, contacts, products and locations.

The purpose of a reengineered ODS, coupled with an EIP, would be to aggregate the required transaction data into a central store, making the data available to more users and applications across the enterprise than is possible with a traditional placement of an ODS or data warehouse. This new positioning would address the wider problem of availability: If any one of the operational systems goes down due to maintenance, upgrade or performance problems, the ODS would give EIP users and applications access to the integrated selection of operational data. Business requirements would dictate how much transactional data the ODS would store; the amount might be similar to that for a traditional decision-support ODS.

ODS and Integration Middleware

Business requirements would also determine how "just in time" (JIT) the data integration would need to be. Organizations could establish JIT data integration by deploying data adapters between the ODS and enterprise operational systems, as shown in the diagram. Enterprise application integration (EAI) software from vendors such as webMethods and Tibco are sources of such adapters, although note that these vendors and others are moving quickly to support nonproprietary standards that function within Java and/or .Net environments. Full-fledged EAI tools provide advantages of scalability, failover, security and guaranteed delivery in both synchronous and asynchronous modes.

You could also look at other current data integration technology to link the ODS with operational sources, including extract, transform and load (ETL) and enterprise information integration (EII) technology. However, remember that an ODS is designed to integrate information coming from all relevant operational systems though extraction and replication. This is fundamentally different from an EII approach, which unifies views of systems at a metadata level, and typically sends queries out to data sources, only bringing back result sets. While your business requirements will guide you to choose the most appropriate technology for your organization, the centralization route is best suited for the "ODS reinvented" approach I'm describing here.

In addition to a centralized database, the repositioned ODS would have a message broker that serves as a distribution hub for messages that carry data and establish sharing between the ODS and operational applications. Here again, availability is better because the ODS database can maintain the data for the EIP. You could use the same EAI approach to replicate transaction data into the ODS itself. In some business scenarios, transaction data is crucial for integrating multiple operational systems; two good examples would be a multiple-order entry system and a set of regional operational systems for human resources management.

Operational systems could use the message brokers and bidirectional data adapters to share core data with other systems. A new item in the order management system, for example, could also update customer information in the billing system. The ODS wouldn't serve as the system of record for core data: That responsibility remains with the source systems. The ODS would serve as an integration point, from which updates could be sent from one operational system to another via the EAI architecture I've described here or through alternative data integration tools. EAI tools would provide message routing, delivery rules, monitoring and support. Business and IT managers would set the data governance rules that determine update frequency.

The primary purpose of this repositioned ODS is information availability. The data warehouse would assume responsibility for all reporting. It would continue to extract data from the source operational systems. However, the data warehouse could take advantage of the data already integrated within the ODS to cut down on sourcing and integrating steps.

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
1 of 2
Comment  | 
Print  | 
More Insights
InformationWeek Is Getting an Upgrade!

Find out more about our plans to improve the look, functionality, and performance of the InformationWeek site in the coming months.

How CIO Roles Will Change: The Future of Work
Jessica Davis, Senior Editor, Enterprise Apps,  7/1/2021
A Strategy to Aid Underserved Communities and Fill Tech Jobs
Joao-Pierre S. Ruth, Senior Writer,  7/9/2021
10 Ways AI and ML Are Evolving
Lisa Morgan, Freelance Writer,  6/28/2021
White Papers
Register for InformationWeek Newsletters
Current Issue
Flash Poll