Information technology professionals are looking to Web services to promote the reuse of expensive and critical enterprise software resources. Spurred on by the success of early adopters, system architects in diverse industries are considering service-oriented architectures (SOAs).
SOA techniques may be relatively new, but deployment goals are the same: to make the best use of development resources, avoid duplication of effort, ease integration with business partners and publish transactional interfaces across business and network boundaries. In pursuing these goals, organizations must make important decisions about how to manage these new software assets.
SOAs are revolutionizing the way businesses integrate systems, departments, divisions, customers and partners. When successful, an SOA represents a large, complex business system, with all the associated management challenges. With hundreds or perhaps thousands of components written by different teams, in multiple languages and running on diverse platforms, the most pressing problems are empowering developers, maintaining the integrity of production systems and measuring performance in a distributed environment.
Fortunately for IT managers, service-oriented technology has matured along with a growing set of tools for managing SOA deployment. Known as Web services management systems, the best of these suites address technology management needs at critical points in deployment. These needs include managing component development and testing; monitoring, controlling and securing distributed networks of SOA components; and ensuring network resilience across generations of change.
Master Development and Testing
Much of SOA's benefit lies in standardized messaging interfaces that reduce dependencies between components. SOA also frees development teams to work in the environment — and with the language — that fits their needs. With this freedom, however, often comes a proliferation of development tools that must be supported. Any wide deployment of Web services management must fit painlessly into the average developer's work patterns. All the major product offerings include Web tools for developers to move components into and out of testing environments, capture messages and responses, and profile component performance. At least one vendor, Amberpoint, has announced an agreement to integrate its developer toolset into Microsoft's Visual Studio development environment, so these functions will be available in the same coding and testing interface.
Developers must be able to check Web services components into and out of testing environments. Consider the average developer work cycle: A component's code is checked out of a source-control database, modified, built and then run locally to check for correct operation before being checked back in. Web services components can be built and tested on the developer's machine, but the entire system can be tested only from a "sandbox" environment in which components from various individuals or teams can be tested together. Developers need front-end tools that facilitate the migration and testing cycle.
Whether testing locally or in a dedicated environment, developers must call functions and verify results, something that hasn't changed since the earliest days of procedural programming. The Web services component interface is a set of XML messages and responses. To test these components, you must construct and send messages to them and capture the results for analysis. Web services management tools construct test messages, validate them against the appropriate schema, dispatch them to the designated server and capture and browse the results. More advanced capabilities include automatic generation of compliant test messages, either with random data or data generated according to editable rules.
Ensuring correct operation is only one aspect of testing a Web services component. Performance profiling is often just as important because large mission-critical systems may handle high message volumes. Existing developer tools are sufficient for local binary component performance testing, but once the Web services are deployed into a test environment, performance data must be gathered there and reported back. This approach leads to an agent-based architecture in which monitoring tools run on the application servers to collect performance and other data. Most Web services management offerings follow this architectural pattern. In fact, as I'll show, performance data is only one example of information that remote monitoring agents can gather.
Move Into Production
Web services systems must be able to survive deployment into highly regulated production environments in which developers will have little access to or control over configuration. At this point, management needs oversight and dashboard capabilities that help map system performance and utilization to business goals. The typical Web services deployment, even with only moderate utilization, generates huge amounts of data that can be stored and analyzed by company decision-makers. In addition, developers must have access to data on problems that occur, and they'll need mechanisms for version control and change migration.
As with earlier technologies such as enterprise databases, IT managers must know how the Web services they've deployed are performing. Metrics include message volume, volume by client and organization, average response time, and number and character of alerts. Easy data collection and analysis are critical. Here again, Web services management tools exploit agent-based architectures to place monitoring capabilities in the run-time environment and provide reporting tools to roll up performance data into actionable intelligence. Data is typically stored in standard SQL databases for archiving or to provide input for performance-over-time analysis and business processes.
Real-time event notification is just as important as aggregate data collection. Even a well-engineered Web services component will sometimes misfire, either through internal errors or when encountering some message it can't process correctly. Managing these errors requires agents to trap system events and exceptions on the application servers while also capturing messages and responses for later analysis. All current enterprise-level Web services management tools provide some of these capabilities. The more powerful suites include rules-based exception-handling and problem-notification workflows. Various real-time alerting strategies can be implemented, including e-mail notification and Simple Network Management Protocol (SNMP) traps.
Several factors make securing a Web services infrastructure challenging. First, the messages are text-based and self-describing, so anyone who can send an HTTP request can technically connect to an interface. Second, the network that feeds messages is typically an IP network with connections beyond the enterprise. Third, the messages are specified according to a schema published to potential system users. In such an environment, IT security staffers need fine-grained control over access rights to specific messages, individual services, application containers, servers, directories and network segments. Most Web services management suites provide comprehensive security management. Those features may simply wrap the existing operating system for access control and user privileges, or they may add a proprietary layer of controls that extends into areas that operating systems don't directly address, such as schema publishing.
Finally, you must look at change control. Web services should be self-describing, and the components should operate in accordance with what the schema and the business logic promise users. Schemas change, and the underlying components evolve either with the schema or in response to system problems. It's essential to have a change management system for underlying software assets, as well as for the schema describing the system. Existing change management software is sufficient for controlling the evolution of code and related binary components, documentation and so on. But additional capabilities may be required to tie all that in with schema changes and ensure that software behavior remains compliant with published message interfaces.
Size Up Web Services Management Tools
The list of capabilities required in a Web services management platform is long, given the distributed nature of such systems. A number of startups developed tools, and now, through a wave of mergers and partnerships, large system software vendors are getting into the game.
Among the acquisitions, Oracle has purchased Oblix and Confluent; Computer Associates bought Adjoin; and Hewlett-Packard acquired Talking Blocks. Among the Web services management vendors that remain independent are Actional, Amberpoint, Infravio, SOA Software (formerly Digital Evolution) and Systinet. Actional, which began life with the acquisition of Canadian software company VisualEdge, has since merged with security vendor Westbridge, and it continues to sign important customers such as Starwood Hotels. Systinet offers a leading UDDI registry, for which it recently landed an OEM partnership with Oracle. Amberpoint will integrate its toolset with Visual Studio. SOA Software has forged deals with IBM Global Services and JetBlue. Whether these and other independents can resist the giants remains to be seen.
A complete review of the products offered by any one of these vendors is beyond the scope of this article, but it's worth examining one suite in detail. I chose Amberpoint for no reason other than its integration deal with Microsoft, which may spark adoption by developers. Like the other pure-play vendors, Amberpoint designed its product to do Web services management from the ground up.
Naturally, Amberpoint's suite is built on agents. The agents sit on developer workstations, on application servers, in network operating centers, and on policy and directory servers. These agents provide much of the system's low-level functionality by interceding between clients and services to apply policies, collect data, monitor version conflicts and generate alerts, among other functions. On top of the agent network, Amberpoint layers four architectural components that provide the rest of the system's functionality. These are the Management Foundation, Service Level Manager, Exception Manager and Amberpoint Express.
The Management Foundation provides basic policy management and enforcement, service brokering, version control, security, schema management, service discovery, dependency analysis and data collection that the rest of the architecture depends on. Like all the Amberpoint components, it's implemented in both J2EE- and .Net-compatible versions, so it will fit into most IT environments. Service Level Manager is the system's IT business information and analysis component, providing tools for defining service-level agreements at different architectural layers and monitoring compliance. Amberpoint's agents generate data on message traffic, error conditions and response times and provide many other metrics. The system stores all this data in any standard relational DBMS, so it's fully accessible for custom analysis and reporting.
Amberpoint Exception Manager handles system, application and business-level exceptions. This component provides local exception trapping, event and message logging, alerting, root-cause analysis, impact analysis and a number of other features that monitor what's happening across a widely deployed SOA. Lastly, Amberpoint Express provides the system's developer workstation component, letting programmers use a native Windows or Java GUI to debug, profile, tune and migrate their locally developed components. Amberpoint Express will be integrated into the Microsoft Visual Studio environment and is available as a free download from the Amberpoint Web site.
IT managers investigating this market will find that the Actional and SOA Software solutions follow similar architectural patterns and are as complete.
Beyond the Pure-Play Vendors
Large vendors such as IBM have built their Web services management tools, largely through acquisition. Thus, these offerings may be less complete than those of the independents, but expect this state of affairs to change rapidly as the heavyweights infuse dollars into research and development. There are also likely to be open-source alternatives in the future. Synapse has submitted a proposal for an SOA architecture to the Apache standards body, but this proposal is moving forward slowly. In the meantime, organizations such as Verizon have developed their own architectures from available commercial and open-source offerings. Verizon's internally developed "IT Workbench" provides all the functionality discussed here and more and is brokering millions of service requests daily for the telecommunications giant.
While most in the industry now agree on what "Web services management" means (and the folks at Apache believe they can standardize the definition), we'll continue to see a variety of approaches over the next five years. In such a rapidly evolving marketplace, IT managers might just wait for the dust to settle. However, in an environment in which audit controls and system manageability are receiving more attention than ever before, the number of SOA deployments is increasing. You can either buy a complete suite from an independent vendor, or you can opt for a slightly choppier toolset from a large player, knowing the vendor will be in the business for years to come. A third option is to follow Verizon's lead and integrate your own solution from available components and custom-built tools. Given the opportunities and inherent challenges in SOA deployment, doing nothing is probably the only wrong choice.
Mark Betz is Vice President of Applications and Architecture at PDI Inc. in Saddle River, NJ. Write to him at [email protected].
.svg)
