The Elephant Has Left The Building

For more than a decade, packaged enterprise applications have ruled. Business reengineering dogma of the early 1990s merged with paranoia about possible Year 2000 system catastrophes to push many large organizations to embrace ERP and other packaged applications at the expense of "legacy" systems and in-house application development. But with the maturation of Web services, could ERP's reign be coming to an end? Like Elvis, has the "king" left the building?

Frustrated by the cost and difficulty of integrating packaged applications, consolidation has become the big story. SAP and Oracle (with PeopleSoft) are today's consolidated applications royalty. Through acquisition and internal development, they have amassed soup-to-nuts portfolios that cover manufacturing, supply chain management, human resources, financials and CRM. Microsoft is angling for similar hegemony among small and midsized business customers.

But the relief from heterogeneous-application integration didn't come without a price: vendor lock-in, high license and consulting fees and a larger degree of inflexibility. For a while, it appeared that the future held little choice. No longer: Web service-oriented architecture (SOA) and "composite" application development are beginning to push the elephantine packaged applications out of the limelight. Top vendors are scrambling; it's now the turn of Oracle, SAP and lesser enterprise application vendors to reengineer or lose their relevance.

Whether you plan to buy or build, the advent of SOA is having a huge impact on your choices for enterprise applications. In this article, we'll look first at some of the problems created by consolidated, monolithic applications, and why a new approach is necessary. Then, we'll focus on how you should address the key enterprise architecture issues that differentiate the old regime from the revolutionary upstart: composite applications.

Consolidation Blues

Application integration horror stories have driven the consolidation trend. Beginning in 1997, Hershey Food Corp. tried to integrate Manugistics, SAP and Siebel applications to improve order processing. About 30 months and $112 million later, Hershey had to admit failure. With its order fulfillment process broken, the company found itself having to call its customers to find out how much candy they had received in shipments. Hershey chose to consolidate more than 95 percent of its revenue and business transaction processes within SAP systems. The company has since reduced costs and boosted quality and efficiency on its integrated platform for enterprise decision-making.

SAP calls Hershey's experience a consolidation success story, saying it's instructive about what most companies face when they attempt to integrate their applications, packaged or otherwise. However, while app consolidation relieves some of the pain, it creates other difficulties.

First, upgrades and modifications to massive applications normally take years of planning and millions of dollars. Experienced SAP integrators would tell you that Hershey dramatically underestimated the effort required to install and customize six SAP modules (finance, purchasing, materials management, warehousing, order processing and billing) and integrate them with two Manugistics modules (planning and scheduling) and one Siebel module (pricing promotions).

Second, what happens if the way in which the consolidated application implements business tasks doesn't match the organization's existing procedures? The company must either spend big money to customize the application's modules or change business functions to what the application specifies.

Third, a company's business processes don't end at organizational boundaries. Businesses are increasingly focused on modeling and implementing end-to-end processes that interoperate with those established by customers and partners. The monolithic nature of the consolidated application portfolio may inhibit such external process integration.

And fourth, companies define their success on their ability to respond to market changes quickly and flexibly, as they grapple with changing economics, new regulations and shifting competition. Monolithic applications bog down already difficult efforts to adapt to new circumstances.

Oracle, SAP and other packaged application providers are no dummies. They're racing to incorporate standards-based integration and SOA into their platforms. Application development and integration vendors such as BEA, IBM, SeeBeyond, Sonic and Tibco sense an opportunity to gain from the growing desire to usher the "elephant" out the door and replace its hulk with a new vision of applications that promises lower cost through standards, reusability and incremental, plug-and-play growth.

Composite Applications

By the mid-20th century, manufacturers knew how to reduce costs and improve time-to-market by building products from reusable parts rather than custom designing all parts for each and every product. The composite application approach applies this lesson to software systems. Supported by SOA, software's reusable parts are services; organizations "compose" application functionality out of a set of shared, reusable services. The goal is to let companies assemble, modify and reassemble these services quickly and easily.

In contrast to big, all-at once application implementations, composite application portfolios come together incrementally based on business need. Analysts define executable business processes, which enterprise architects and developers then turn into composite applications. With SOA, process flows that typically run into trouble when they must touch multiple applications have an easier time working with standards-based architectures. There will also be less need to duplicate tasks of individual applications that may be supplied by reusable services.

Sarbanes-Oxley Act (SOX) compliance is one such need that will drive service-oriented, composite approaches in some organizations. CFOs might find that they have multiple departments or divisions implementing nearly identical systems to address SOX requirements. Shared services used by multiple groups could save millions of dollars in development and consulting costs, and improve the quality and control of governance efforts.

SOA should be of great interest if you have sizable IT investments. Can you use the services approach to more easily integrate legacy systems, thereby bringing them into the incremental, composite fold? Merrill Lynch needed to build a new set of Web-based financial applications to support its much-expanded worldwide force of financial analysts. SOA and composite application development allowed the company to deploy Web applications that can tap into its portfolio of more than 40,000 mainframe-based systems running with IBM's CICS Transaction Server software.

Ultimately, the composite approach encourages IT to align itself better with business objectives. In the highly volatile mobile telecommunications industry, for example, vendors compete by offering new and intriguing services and content. Most of these come from third-party providers. Vodafone and T-Mobile rely on SOA to manage provisioning, accounting and billing of consumer usage of the third-party services and content deployed on their wireless networks. In this way, IT meets business objectives by enabling — and being directly responsible for — customer — and partner-facing services.

More than Middleware

Most discussions of SOA focus on it as an expression of the Web Services Framework (WSF), the evolving XML-based protocol effort led by W3C to support automated definition, deployment and manipulation of decentralized services. Nearly all application platforms, infrastructure services and purchased systems support WSF. This foundation for SOA is critical, but WSF implementation is not all there is to successful SOA and composite applications.

Composite application assembly starts with creating a portfolio of services. The services' design must support both assembly and reusability, not just the WSF duties. Enterprise architects and developers must change their behavior, in other words, to push SOA beyond middleware to define a set of design principles and best practices that further a key goal of composite applications: reusability.

Your SOA, then, should establish application functionality as a set of shared, reusable services. Where appropriate, an enterprise service bus (ESB) would be the technology choice for the middle layer of the SOA. Playing a role that extends what's been done with message-oriented middleware into the realm of Web services and XML, ESB software is offered by pure-play vendors such as Cape Clear and Sonic Software, and will be by established integration providers such as BEA and IBM.

Each service must carry out a discrete business task. Any application that needs to perform that task should use the shared service to do so. The vision is that when business rules associated with a task change, developers modify only the one service that performs that task. All applications that use the service will automatically adopt the new business rules. Developers can respond to new business processes by assembling services into a new, composite application.

Reusability has been a Holy Grail of software development forever, and a priority of object-oriented methods. However, whereas object-oriented applications typically share code but expect each application to execute its own instantiation of the code, SOA systems share a single code instantiation: that is, a service, at runtime.

Another important difference between services and earlier object-oriented efforts involves granularity. With object-oriented methods, the shareable component is an object, which often has an application-specific focus rather than a business focus. Composite applications use services that represent business tasks as their shareable components.

Design: The Mental Shift

Historically, enterprise architects and developers built software with particular applications in mind. As a result, functionality matches specific user interfaces and is limited to a defined set of use cases. Application-specific orientations aren't conducive to reusability. Composite applications disassociate business functionality from any particular application, use case or user perspective.

In application-centric paradigms, architects and developers focus on building specific applications and factoring them into logical tiers. Usually, developers expect to create an entire application. SOA demands that developers focus on building reusable services and then assembling those services into composite applications that implement a business process. A given service may show up in any number of applications. Developers should write only as much code as is necessary to orchestrate the business process.

Let's look at some basic design principles and best practices for producing loosely coupled, reusable services that become the ingredients of successful composite applications.

Loose coupling is a primary design goal of service orientation. Consumers interact with a service through its interface. Implementation details — such as the programming language, operating system platform and internal object model — are hidden. The loosely coupled design objective is to avoid dependencies among systems other than through the service interface so that composite applications can easily bring together disparate services.

The service should live at a higher level of abstraction, cleanly separated from implementation details. This is essential not only to enable composite application flexibility, but also to free IT to maintain applications behind the scenes. Developers can then modify the application without affecting the consumers of the service.

Enterprise architects and developers should follow three rules when considering how to raise the level of service abstraction above application implementation details:

Another important goal is to ensure versatility. Many applications will share the same instance of a service, but also in many cases, each application will require different operational semantics that are important for transaction security and other essentials. In conventional applications, you will normally find operational semantics within the code itself; this makes reusability almost impossible.

Composite applications must have the business logic defined separately from each application's infrastructure. For example, instead of baking security semantics into the code, use an external, general-purpose security framework. This will be your template for enforcing authentication and authorization rules based on declarative policies that specify the required security semantics for the service within the context of a specific application.

SOA Needs IT Governance

Guidance is essential for composite application architects and developers to feel secure in adopting SOA. To be successful, training must link up with ongoing IT governance to guide service development and deployment.

Governance refers to the processes an enterprise puts in place to make sure things are done right — that is, in accordance with best practices, architectural principles, legal and industry regulations and other factors. Governance can give a business context to guide the fundamental changes in design, development and project management that must occur if SOA and composite applications are to take off. People are naturally resistant to change, so a governance framework that expresses the new bylaws and goals is essential.

CIOs and application development managers must also look at incentives. Do current incentive programs actually hinder the development of reusable services? Most programs focus on time-to-market and cost containment. Designing for reusability takes more time and resources. The development team that implements a reusable service often derives no immediate benefit. Why should a developer, much less a line of business manager who has budget for development, agree to accept the increased burden of developing reusable services just so someone else can benefit?

Control is another disincentive that IT management must address. A line-of-business manager won't want to expose his objectives to missed schedules or project disruption because of another group's failure to deliver a service on time. Also, some groups prefer to reinvent the wheel because they're certain they can do it better.

Governance can help shift the culture. When no existing service is available, developers need incentives to create services that other developers can reuse.

As expectations rise about the potential of SOA and composite applications to bring down costs and deliver higher quality to business processes, companies will question the wisdom of monolithic packaged applications. The age of the massive, lumbering, elephantine application is passing. The new age of flexible, plug-and-play services is about to begin. With careful planning and attention to both technical and cultural details, your organization will grasp the new reins of IT power and ride to market first.

Anne Thomas Manes is VP and research director of Burton Group, an IT research and advisory firm. She is a 24-year industry veteran and expert in Web services and distributed computing technology. Write to her at [email protected].