IBM recently announced a unified metadata services infrastructure that's designed to ease metadata management, access, and sharing within a service-oriented architecture (SOA). The IBM WebSphere Metadata Server will be generally available later this year as part of the Hawk release of the IBM WebSphere Information Integration (WII) platform. WebSphere Metadata Server provides metadata management as a service to products in the WII platform and provides a common metadata services infrastructure for metadata initiatives in other IBM software brands.

SOAs have attracted a lot of attention over the past two years, and many expect 2006 to be the year when SOAs move from pilot programs to mainstream production. Analysts predict that in the next two years SOAs will provide the basis for the majority of new development projects.

SOA owes its primary benefits (flexibility, ubiquity, and reusability) to the role standards and metadata play in its design. Many information management projects similarly leverage metadata to speed development cycles and improve information understanding. In these projects, the value of metadata is magnified when different types and sources of metadata are linked together and reconciled.

As the role of information within SOAs continues to grow, so, too, will the importance of the connections between traditional metadata management techniques and these new metadata-driven architectures.

Changing the Game

Today's businesses need to be agile. Change comes from every angle — mergers and acquisitions, new regulations, competitive pressures, deeper partner collaboration, and even outsourcing. Unfortunately, most applications aren't designed for flexibility. Instead they're hard-coded to perform specific business functions in specific ways, making it extremely difficult for them to respond to new requirements. An SOA allows specific information or functions in these inflexible applications to be made available, outside of the confines of the systems into which they were originally built, in an uncomplicated and standardized way. These blocks of functionality and data are called services. Services can be reused in new ways by new processes, so the business becomes adaptable to change, even when the original systems weren't.

An SOA removes the complexity of the languages, platforms, locations, and formats of source systems and allows creation of easily reusable services that are based on universally adopted standards. In fact, services commonly span multiple systems. For example, a single service could provide a comprehensive view of customer data across a data warehouse, a campaign management system, and an ERP system. The service removes the burden of finding and integrating this information from the application developer. The developer simply reuses an existing service that provides standard customer information, without worrying about how the information was assembled. In fact, the sources behind the service could change, and the developer and the application wouldn't be affected.

Although most people don't realize it, the foundation of the flexibility and openness in an SOA is actually metadata. Metadata enables services to abstract the complexity of source systems. Metadata also makes services easily reusable and interoperable. In essence, SOA's greatest benefits are achieved by escalating the role of metadata in the design of business functions.

What Is Metadata and Why Does It Matter?

Metadata is defined in many ways; the most common definition is "data about data." More accurately, metadata is data that describes processes, information, and objects. Those descriptions address characteristics such as technical attributes (for example, structure and behavior), business definitions (including vocabulary and taxonomy), and operational characteristics (such as activity metrics and usage history). Metadata encompasses so much that it's commonly misunderstood and seldom fully exploited.

But metadata is critical. In fact, it's the underlying enabler of a whole new wave of software innovation. Until recently, most metadata was used to describe technical things to people. Today, metadata is increasingly used to describe technical things to other technical things — removing manual translation steps entirely. For example, schema metadata from modeling tools can be used to automatically generate database load routines in an extract-transform-load tool, removing the need to manually build these routines. SOAs provide an excellent example of this trend, leveraging metadata to automate discovery, contract, and communication of services.

SOAs rely on three primary standards, each of which is a highly standardized set of metadata:

• Simple Object Access Protocol (SOAP) describes the runtime call to a service — what data to pass in, which method to call, which security credentials to apply, and so on.
• Web Services Description Language (WSDL) describes the interface of the service — which methods are available and which schema each method expects as inputs and outputs.
• Universal Description, Discovery, and Integration (UDDI) describes all the services available and where they can be found.

Because this technical metadata is very standardized, it's machine-readable, meaning that Web service consumers can simply read a WSDL file and structure a SOAP request in the appropriate format to call the service. This is the secret to why services are so portable, interoperable, and flexible. The metadata describing the interface and the service call are completely standardized and abstracted from the implementation. This standardized metadata allows services to remain platform and language independent and allows the sources underlying the service to change without requiring the applications or processes that call the service to change.

Metadata in Information Management

Metadata is a more familiar tool in the information management arena than in other parts of the IT organization. Technical metadata, such as schema, is commonly used in daily routines to understand and work with source systems. Data profiling tools such as IBM WebSphere ProfileStage can extend schema metadata with more detailed information gleaned from looking at the actual data contents, a technique often used to accelerate information integration projects. For example, IBM WebSphere DataStage can see and leverage metadata discovered using WebSphere ProfileStage, to speed up or even automate the design of information flows.

Business metadata is also commonly used by information management groups; the metadata applied to index image files is a good example. Business metadata is typically managed and maintained by business analysts, data stewards, and subject matter experts to provide business definition around information. For example, a finance department may have a specific definition of corporate profitability. Business metadata is often complicated by the fact that a complete business definition often requires an understanding of how things are related to each other (in taxonomies and hierarchies); different parts of a business often have different definitions for the same term.

When Metadata Worlds Collide

Information management teams are increasingly using business metadata to communicate with business and IT teams in a way that shrinks the cycles between specification and design. Linking technical metadata about the structure, contents, and location of information with business metadata about the meaning and context helps IT respond to changing business requirements. For example, if a new regulation requires reporting on corporate profitability, and business and IT teams already have a clear understanding of what that profitability means and where the required information can be found, then it's much easier and faster to design a compliance solution.

This connection between business and technical metadata magnifies the intrinsic value of metadata. With this connection in place, for example, it becomes easy for business users to trace the lineage of a piece of information back through the rules and source systems from which it was derived — a common requirement for many compliance projects.

As the role of information within SOAs expands, the value of the connections between business and technical metadata is becoming more apparent. When trying to make a decision on which service to use, for example, developers need to understand where the information within the service came from and how the business defines that information. Likewise, it's important to know beforehand which services may be affected by a change in a data source. In order to derive this insight, service registry metadata needs to be linked with the technical and business metadata underlying the information. With this linkage in place, services are not only easier to manage, but also more valuable to the business — providing confidence and insight in addition to information.

WebSphere Metadata Server is designed to manage these types of metadata interconnections. It provides management, analysis, and metadata interchange within and outside the WII platform and allows business metadata to be created and maintained within the same repository through WebSphere Business Glossary. These features preserve the link business, technical, and operational metadata. This metadata linkage even extends to services deployed from the platform, allowing developers to find valuable information on where information comes from, how it's processed, and how the business defines the information, prior to selecting a service. WebSphere Metadata Server, which will be released later this year, forms the foundation of the IBM metadata strategy.

Key Relationships

Metadata has long been a critical tool for information management professionals. Today, metadata's importance is expanding beyond information management, particularly as metadata-driven architectures such as SOA are emerging. This metadata expansion is a boon to information management teams, DBAs, and information architects, who have a working understanding of metadata's value and the experience to exploit it in projects.

The potential of metadata is only truly realized when the relationships between sources and types of metadata are understood. SOA strategies should include linkages to traditional metadata management activities to fully leverage these connections and maximize the business value of the organization's information assets.

---

Michael Curry is the SOA marketing lead for IBM Information Management and a product marketing director with IBM Information Integration Solutions. He is a prominent speaker on Java and XML topics and a frequent contributor to industry articles and publications.

---

Resources

WebSphere Information Integraton