Welcome to my new column on application integration issues. I'm excited to share my thoughts on this topic, and I'm looking forward to interactions with all of you, so please feel free to email me. If you've been following my application integration work for the last several years, you'll find that this column is more innovative and forward looking, written for those of you that have to make decisions now and understand the strategic nature of this technology. So, let's get started.
Have you ever heard the phrase "If you sell a hammer, everything looks like a nail?" That's a good description of how many people approach application integration today, looking to a single technology or standard to solve problems that typically need a very different or more complex technology solution.
The fact is, application integration, for all but the smallest and most simplistic organizations, is difficult and complex, typically requiring in-depth requirements gathering and a group of complex technologies applied in specific ways to create the proper solution. However, many people still hope to find that magic bullet that will meet all their application integration needs, forcing a single solution pattern, technology, or enabling standard as a cure-all. I'm not sure such a thing exists. Let me explain why.
The trouble with the magic bullet theory is that the source and target systems, complexity of the interfaces, and most business requirements are all very different from enterprise to enterprise. Thus, proposing a single solution, be it Web services or messaging, could quickly get you in trouble. While ABC Corp., a logistics company, is focused on information-oriented integration due to its need to leverage XML and EDI with trading activities, XYZ Corp., a financial services company, is focused more on service-oriented integration due to its need to leverage financial application services (that is, risk analytics) found in its local and remote systems.
What's more, you also need to consider the subdomains, groups within the larger domain that may have different requirements, as well as the limitations of the interfaces provided by the applications. For example, although some application interfaces provide access to services (such as Web services interfaces), most only produce and consume simple information. As a result, leveraging Web services as your application integration solution won't do you much good because the services aren't exposed.
You can make better sense of your requirements by understanding the common integration patterns or approaches that are applicable. That is, once you understand your business requirements as well as the properties of the source and target systems present, you can break these patterns into basic categories: information-oriented, service-oriented, and process-oriented integration, as well as hybrids leveraging two or more approaches.
Information-oriented integration is required for those problem domains that need only replicate information between two or more systems. Truth be told, most of the organizations I deal with only need to leverage information replication to meet their requirements. Information-oriented integration is less expensive and complex than the other forms of integration because you're merely extracting information out of a source system (an application interface or database), transforming it to account for the semantic differences, and placing it in a target system. With information-oriented integration, you don't have to deal with application behavior, which increases the cost and complexity of an application integration project to a large degree. In other words, don't go service-oriented unless you need to.
Technology applicable to information-oriented integration include message brokers and integration brokers from vendors such as SeeBeyond, Mercator (now part of Ascential Software), and WebMethods; messaging middleware such as IBM's MQSeries or Sonic Software's SonicMQ; database replication servers; and other technologies that deal with the replication of information between two or more systems. Integration technology that's contraindicated when it's a pure information-oriented requirement includes Web services, CORBA, and application servers.
Service-oriented integration is required for those problem domains that need to share both application services and information. This approach provides the infrastructure that lets applications leverage behavior through service access from other applications, creating a composite application. The idea is to use application services that already exist vs. recreating them over and over. If done properly, this approach can reduce maintenance costs and make the outcome of the application integration project that much more valuable. Assuming, of course, there is a legitimate need. If it isn't needed, it only serves to add significant and unnecessary cost to an integration project.
Although distributed object standards such as CORBA and COM used to be the most popular way to approach service-oriented integration, today, most organizations are opting for Web services (which have design patterns consistent with traditional distributed objects, by the way). Application servers, such as BEA Systems' WebLogic and IBM's WebSphere also fit here, as do traditional development tools. Basically, any tool that can join applications together so that they're able to handle abstract behavior between applications fit into this category.
Process-oriented integration, in its functional form, provides you with the ability to join internal application processes together to create a master or meta process that exists to bind the applications together (see Figure 1). For example, you may have a single system that contains the processes associated with creating a product, another with processes associated with selling a product, and still another that ships a product. Process integration would bind these processes together, automating how the systems work concurrently and, in essence, creating a master process that spans many systems.
The difference between this tactic and information-oriented integration is that you approach process integration by creating a common business process to bind the applications together, whereas the information-oriented approach connects the applications through a series of information flows. Indeed, process-oriented integration and information-oriented and service-oriented approaches often coexist. The process integration technology provides another layer of abstraction (or another way to represent the information flows or service invocations), orchestrating how the systems interact or share both information and services.
Process integration technology exists in two forms: process integration technology from traditional EAI players such as Mercator and WebMethods and standalone process integration technology from vendors such as Metastorm and Versata. The technology from the EAI vendors is typically bound to the technology they sell, whereas the standalone technology needs additional development to see deeply into the source and target systems.
Like service-oriented technology, pro-cess-integration technology should only find its way into a project when it's needed. Typically, process-integration technology is indicated when the problem domain is complex (more than a dozen systems) and leverages abstract processes (instead of only information flows), or if the problem domain includes a composite application that lets the end users better create and manage their integration solution. The more systems you integrate and the more non-automated processes that exist between your applications, the more you'll need process integration.
In a simple world, all organizations would go either information-oriented or service-oriented, with some leveraging process integration and some not. However, things aren't that easy at all. Most enterprises or trading communities will find that they need to mix and match technology to meet their application integration needs. You could find yourself with several brand names, technology types, and standards. If that happens, don't be concerned as long as you've done your homework and understand your own requirements and the purpose of each technology. Most large enterprises will fall into this category and will have what seems like a hodgepodge of integration technologies but it's probably the proper mix of application of technology. In contrast, many organizations force solutions (such as Web services) where they aren't needed and thus make the project overly expensive and complex, clearly risking failure.
As the world of application integration evolves, we are learning that application integration is hard work that requires a careful understanding of your own requirements as well as the knowledge of which technologies to bring to bear and how those technologies work together to form the right solution for your enterprise. So, the next time you hear your vendor say that they are "one-stop shopping" for all your application integration needs or the .Net zealots down the hall proclaim that all things must be Web services, know that you must first begin with your requirements, and then look for the technology or technologies that work for you. The technology must meet your requirements. Forcing your requirements to meet a technology just doesn't make good sense.
David S. Linthicum [[email protected]] is the author of three books on application integration, including Enterprise Application Integration (Addison-Wesley, 1999) and Next Generation Application Integration: From Simple Information to Web Services (Addison-Wesley, 2003). He is the former CTO of Mercator Software, as well as the former CTO of SAGA Software, both application integration technology vendors.
Ascential Software's Mercator solutions: www.ascential.com/news/mercator/index2.html
BEA Systems: www.bea.com
SeeBeyond Technology: www.seebeyond.com
Sonic Software: www.sonicsoftware.com