Key Differences Between Data Integration and App Integration

There's been a blurring of the lines between data integration in the BI environment and in the operational environment. It used to be that you bought ETL tools for the DW, and mostly hand-coded data integration for OLTP projects.

The problems come when vendors obscure the differences between data and application integration to broaden the appeal of their tools. You'll find EAI and messaging vendors tout their tools for DI, and ETL vendors talk about operational DI.

When evaluating tools, it's important that you realize that data integration and application integration are not the same thing.Application integration focuses on managing the flow of events (transactions or messages) between applications. Data integration focuses on managing the flow of data and providing standardized ways to access the information.

Application integration addresses transaction programming problems, allowing one to directly link one application to another at a functional level. The functions are exposed to external applications via the tool's API, thus hiding all applications behind a common interface.

Data integration addresses a different set of problems. DI standardizes the data rather than the transaction or service call, providing a better abstraction for dealing with information that is common across systems. DI tools abstract the connectors, transport and more importantly manipulation - not just the system endpoints. When done properly, DI ensures the quality of data as it is being integrated across applications.

The type and level of abstraction are what differentiates the two classes of integration. Enterprise application integration (EAI) tools are a transport technology that requires the developer to write code at the endpoints to access and transform data. These tools treat data as a byproduct. This makes functions reusable at the expense of common data representations.

Data integration tools use a higher level of abstraction, hiding the physical data representation and manipulation as well as the access and transport. The tools provide data portability and reusability by focusing on data and ignoring transaction semantics. Because they are working at the data layer there is no need to write code at individual endpoints, and all data transformation and validation is done within the tool.

The key point in differentiating DI and EAI is to know that there are two distinct types of integration with separate approaches, methods and tools. Each has its role, one for managing transactions and one for managing the data that the transactions operate on.

Embedded below is a presentation entitled "How to Choose the Right Tools for Operational Data Integration" that discusses some of the differences between these types of integration, as well as more detail on the requirements and scenarios for operational data integration.

How to Use the Right Tools for Operational Data IntegrationView more presentations from Mark Madsen.

There's been a blurring of the lines between data integration in the BI environment and in the operational environment. You used buy ETL tools for the data warehouse while hand coding data integration for OLTP projects. Problems arise when vendors obscure the differences between data and application integration to broaden the appeal of their tools...