Sponsored By

The Myth and the Magic of Enterprise Information Integration

Those that know what they're talking about see EII as extending the data warehouse -- not replacing it.

InformationWeek Staff

August 20, 2004

4 Min Read

As the relatively new boy in town, enterprise information integration (EII) has attracted some attention. He's powerful and attractive: so quite a few of the young ladies are keen to be seen with him. By the same token, some of the older guys see him as a definite threat.

Among these guys, the more traditional data warehousing experts and vendors have been lining up to take a few pot shots: What about data integrity? Won't this impact operational systems performance? How will it cope with missing data? they ask. Isn't this just the Virtual Data Warehouse come back to haunt us?

Well, clearly these guys are not your typical playground bullies — their questions do point at genuine concerns. Enterprise information integration does indeed allow direct, on demand access to data, wherever it lies. As a result, one could encounter the problems they describe. But let's stand back a moment. There is an urban myth at play here. Those issues would arise only if EII was being used in place of a data warehouse. Now, there may be a few vendors who are promoting EII as a replacement for data warehousing, but those who know what they’re talking about see EII as extending the data warehouse — not replacing it.

So, this is what the young ladies see in EII. If we already have a data warehouse — or some other repository of reconciled, integrated data, such as an operational data store (ODS) — then we can use that store to ensure data integrity, provide the historical view, and then branch out into some interesting and valuable new areas.

Here are a number of ways that EII can extend the data warehouse and bring significant and early business value:

  • Accessing current data in combination with data already in the warehouse environment to get an up-to-the-minute view of the business or understand the last points on the trend curve, and linking directly to modern business process monitoring directions

  • Linking to unstructured data such as a document, photo, or e-mail, that is a valuable and necessary part of the larger decision-making process. Who wants to copy all of their content stores into the warehouse as well?

  • Formally linking Internet data into executive decision support. All executives rely heavily on external information as part of their decision-making process; EII allows us to move from ad hoc retrieval of such external data to making it part of the formal process.

  • Using data from partner, government, and other external systems as part of the decision support process. In many such cases, legal, commercial and other considerations will limit copying such data wholesale into an internal warehouse. EII allows us to go get the relevant data as needed.

In short, EII is a vital component of the real-time enterprise. This is the magic! It allows us to get at extensive and significant information that really has no place in the data warehouse. Now we can avoid bloating our warehouses with unnecessary data. And we can get instant access to the data we really need. But EII does not do away with the warehouse, and it does not replace it: it augments and extends the warehouse. Of course, EII may have an impact on the operational systems. But what we need to recognize is that as business moves increasingly to 24x7, the impact of the traditional Extract, Transform and Load (ETL) approach also becomes an issue. In either case, IT has to find ways to manage in this new world.

And when it comes to data quality, history, and similar issues, it still remains the responsibility of the data warehouse, ODS, or other data reconciliation approach to fix those problems. In fact, the data warehouse or equivalent is the key source used by EII to obtain cleansed and reconciled data and as the basis for creating those cross-system queries.

The question is not about EII vs. data warehousing at all, but rather how can we combine the strengths of both enterprise information integration and data warehousing for maximum business value? Personally, I think the young ladies have got it right! After all, which would you choose — the myth or the magic?

Dr. Barry Devlin was responsible for the definition of IBMs warehouse architecture in the mid 1980s and is a widely respected consultant and lecturer on this and related topics, and author of a comprehensive book on the subject "Data Warehouse from Architecture to Implementation." Barry currently works on DB2 Information Integrator, where he is part of the team defining IBM's information integration architecture as well as industry solutions and applications of the technology. Barry is 20 years in the IT business, mainly with IBM's Software and Solutions Centre in Dublin. He is a Council Member of the IBM Academy of Technology and an IBM Distinguished Engineer.

Never Miss a Beat: Get a snapshot of the issues affecting the IT industry straight to your inbox.

You May Also Like

More Insights