InformationWeek

July 14, 1997

Line Up For Data

Microsoft Message Queue delivers manageable message-based transaction processing to Windows NT

By Alan Dickman and Ken Walker

icrosoft is challenging the dominance of IBM and its MQSeries transaction-processing product in the high-volume TP market. Microsoft Message Queue Server is the company's answer to MQSeries-and Microsoft plans to bundle MSMQ with its Windows NT operating system. l We've found that MSMQ offers several benefits over MQSeries, and has relatively few weaknesses. Most companies rely on TP technologies to develop mission-critical applications. Message queue manager technology, such as MSMQ and IBM MQSeries, makes it possible for applications to reliably send messages to other applications. A growing number of distributed TP applications are engineered to use a deferred or asynchronous processing model-the kind provided by these products. l TP applications communicate by placing messages on queues and removing messages from queues; queue managers manage queues. MQSeries and MSMQ share many features. Both allow an applications' queues to be created dynamically. Both provide a simple application-programming interface (API). And they both support recoverable and nonrecoverable messaging-nonrecoverable messages are lost if a queue manager fails, and recoverable messages survive noncatastrophic failures. l But one area where there is a difference between the products is in administration. MSMQ provides a superior architecture for administering large, complex enterprises.

In MQSeries, all administration centers around individual queue managers. Application messages are sent between queue managers via transmission queues. A transmission queue must be configured between every pair of queue managers that will communicate directly; two transmission queues are required if messages will flow in both directions. This architecture provides a flexible model for routing messages throughout the enterprise, but administering large enterprises can be difficult.

In a fully connected configuration of N queue managers, you must manage N times (N-1) transmission queue relationships. This number can be greater if you want redundancy or lower if you don't need all queue managers to be directly connected.

In the MSMQ architecture, queue managers are grouped into sites. Communication is organized around network connectivity rather than around point-to-point relationships between queue managers. Information about sites and network connectivity is shared throughout an enterprise. Thus, MSMQ can guarantee that messages are routed correctly and optimally. For large enterprises, this management model will scale better than MQSeries' approach.

Another administrative plus for Microsoft's technology is that it has a graphical user interface for managing the MSMQ enterprise, making it much easier to administer. MQSeries has no graphical management interface. And then there's the price-Microsoft recently announced that MSMQ would be bundled with Windows NT. Getting a technology this powerful for free is hard to beat.

MSMQ is also tightly integrated with Windows services. This is important to the growing number of large and small companies migrating servers to NT. Finally, MSMQ imple ments message security based on public-key technology and X.509 certificates. If applications elect to use security, MSMQ handles all encryption and decryption chores.

The most obvious weakness of Microsoft's technology is platform coverage. MQSeries is available on numerous operating systems, including Windows, Unix, and MVS. On the mainframe, MQSeries integrates with CICS and IMS.

MSMQ servers will always reside on NT. However, an agreement with Level 8 Systems will enable AIX, HP-UX, Solaris, DEC VMS, AS400, CICS, MVS, and Unisys programs to act as MSMQ clients. Level 8 Systems Inc. in New York has also developed a connector between MSMQ and MQSeries.

MSMQ is largely untested, as it has still not been generally released. But we were very impressed with the performance of the second test version, and expect to be similarly impressed by the final product.

Diagram: The Microsoft Message Queue Enterprise Sending An d Receiving Messages
To support different application requirements, MSMQ provides three forms of messaging:

express messages, which are stored in memory until they can be delivered and may be lost if a system crashes;

recoverable messages, which can survive a system crash, because they are stored on disk; and

transactional messages, which can be sent or received as part of a larger transactional context.

An application can put messages in queues, remove other messages from queues, and perform database updates. These operations can all share the same transactional context-all queue operations and database updates in the transaction are committed or rolled back together.

Messages are always sent asynchronously-the sending application immediately resumes execution once a message has been submitted. MSMQ attempts to place a message on a target queue when it is submitted or on an intermediate queue if message routing is configured. If this is not possible, it will be sto red locally until delivery is possible.

When reading a message, an application can remove it from a queue or just read its contents. MSMQ applications can synchronously or asynchronously read a message. During a synchronous read, the application stops all other processing until a message is available or the read operation is terminated by the application after waiting for an amount of time set by the application developer.

Three different mechanisms are made available for reading a message asynchronously: A callback function provided by the application developer is invoked, a Windows Event object specified by the developer is signaled, or (on NT only) an I/O completion port specified by the developer is sent a completion-notification packet.

An application can assign a priority (0 through 7) to a message it sends. Messages are presented to the receiving application in priority order; messages with the same priority are presented in the order they were submitted.

If an application needs to u se secured messaging, developers do not need to encrypt or decrypt messages. They set an additional property on the messages to mark them as private.

MSMQ uses RC2 and RC4, cipher encryption algorithms designed for RSA Data Security Inc. in Redwood City, Calif. RC2 is the default. Message encryption is based on public-key encryption using the Microsoft Cryptography API, with underlying RSA encryption code.

Applications can sign messages by attaching a certificate containing the user's identity. Two types of certificates are allowed: external and internal. An external certificate is one obtained from a certificate authority such as VeriSign Inc. in Mountain View, Calif. An internal certificate can be generated for you by MSMQ. Either type must be registered in MSMQ's information store before it will be recognized.

For most operations, MSMQ uses NT-style access tokens to verify access rights; however, it uses the certificate accompanying a message to determine whether the message can be posted to the requested queue. Applications can also use the certificate to make security decisions of their own.

System And Application Queues
Two types of queues are defined in MSMQ: application queues and system queues. System queues are created by MSMQ, and only MSMQ can send or receive messages on system queues.

Journal queues are the most important system queues. Every server and workstation has a journal queue for tracking the messages it sends to other computers. Application queues also have journal queues for tracking messages removed by applications.

Application queues can be created dynamically by your messaging applications or set up by administrators. Applications use these to send messages to other applications, receive response messages, and receive message delivery acknowledgments that are generated by MSMQ.

An application queue can be created as a public queue or a private queue. Applications can use MSMQ lookup functions to locate public queues anywhere in the enterprise. If an application creates a private queue for other applications to use, it must pass the location of the private queue to the other application.

Applications can be written to send messages to a specific queue, or they can search the enterprise for any queue that matches some set of restrictions. Multiple restrictions can be specified for a single search.

In addition, the possible values of one queue property, Type, are meant to be specified by application designers. For example, queues for printers might have one value to indicate a PostScript printer, a different value for a color transparency printer, etc. An application that wanted to print a PostScript file could include a restriction on the Type property during a queue search to locate all the PostScript printer queues.

Scalable Architecture
MSMQ offers a cohesive architecture for organizing machines, networks, and queue managers. It also provides services for keeping this information up to date. This is a key reason why large, distributed MSMQ infrastructures will be easier to administer.

All computers in an organization running MSMQ are part of an MSMQ enterprise. They share a common distributed database called the MSMQ Information Store (MQIS).

An MSMQ enterprise consists of one or more logical groupings of computers, called sites. Computers within a site do not have to use the same network protocol or communicate directly. But it is assumed that communications between computers within a site is fast and inexpensive.

Sites are connected to other sites through site links. Administrators set the cost of routing messages over site links. MSMQ determines a message's inter-site routing by selecting the minimum-cost route.When multiple site links are available, MSMQ provides load balancing and failure recovery.

Each site has a Primary Site Controller, which maintains the master copy of information about the computers and queues in that site. One PSC also functions as the Primary Enterprise Controller, maintaining th e master copy of information about the enterprise topology, site links, connected networks, and user settings. The site and enterprise information is used to route messages within sites and between sites. Multiple Backup Site Controllers can be installed for each site.

MSMQ defines three kinds of hosts: servers, workstations, and clients. MSMQ servers allow access to enterprise and site information. They can also route messages to target queues and act as queue managers. The Primary Site Controllers, Primary Enterprise Controller, and Backup Site Controllers are MSMQ servers.

MSMQ workstations are similar in many respects to servers. In addition to sending and receiving messages, they can create local queues. This can reduce the number of servers required to support messaging across an enterprise. But workstations cannot act as MSMQ routing servers, and they must access MQIS through a server.

MSMQ clients rely on an assigned server to perform functions-such as queue creation-on their behalf.

Connected Networks
MSMQ organizes low-level communications between computers through "connected networks." Computers that support the same network protocol (IP or IPX) and can communicate directly with each other are on the same connected network. If your company operates two connected networks, a server must be placed on both connected networks to facilitate communication between them.

MSMQ provides flexible routing of messages within sites and between sites. MSMQ always chooses the shortest available path as measured by the number of servers a message must pass through to reach its destination.

Within a site, In Routing Servers (InRS) and Out Routing Servers (OutRS) can be assigned to a workstation. If a workstation is configured to use these servers, all outgoing messages are sent to the Out Routing Servers, and all incoming messages are routed through the In Routing Servers.

Queues can be assigned priority values between -32,768 and 32,767. MSMQ routes messages based on queue p riority first, and message priority second.

Using the MQIS distributed database, servers can reliably and optimally route messages to their target queues. The database is replicated immediately if the site topology is changed-for example, when a site is added or deleted. Administrators can specify intervals for replicating MQIS data within a site and between sites.

Programming Interfaces
In addition to standard C/C++ functions, MSMQ provides ActiveX components. These components cover the most common MSMQ functionality, but that's not quite everything. For example, the ActiveX control's queue lookup method limits you to a small number of restrictions on the search, and not all of the message properties are available through the MSMQ Message component.

MSMQ provides two interfaces between applications and E-mail: the Messaging API (MAPI) Service and the Exchange Connector. Both provide the same programming interface, MAPI, to allow E-mail-based applications to send messages to and receiv e messages from MSMQ queues, but they are implemented differently. The MAPI Service requires MSMQ to be installed on each machine and uses locally configured address books, while the Exchange Connector only needs to be running on one machine and shares one global address book among all machines. However, the MAPI Service can communicate with any of a number of E-mail servers; in contrast, the Exchange Connector works only with Microsoft Exchange.

We ran across two problems in our tests, but consider them to be minor given MSMQ's current status as a test release. Although the release notes indicate MSMQ should work with NT Service Pack 3, we were unable to get MSMQ to install on top of it. This is an understandable situation considering the MSMQ beta was shipped several weeks before SP3. It did run on machines with SP2.

We were unable to generate internal certificates on NT machines. Presumably one could still use external certificates for secure messaging, but we did not test this. We were able to gen erate internal certificates on a Windows 95 machine. Microsoft is aware of these problems and is addressing them.

Conclusions
MSMQ is an attractive product. It has an architecture that should scale to very large enterprises with minimal increases in operational complexity. A variety of deployment options seems viable, from clients being used on handheld devices to workstations that operate in disconnected mode.

Most application developers should not find the MSMQ programming interface to be complex. However, most programmers who are already familiar with OLE and ActiveX may choose to leverage MSMQ as a selectable transport underneath Microsoft's Distributed Component Object Model (DCOM). This combination offers users the value of an object model supporting reliable and transactional messaging.

Users should take a serious look at Microsoft Message Queue Server-whether it's used through ActiveX and DCOM or the native interface-if they need a distributed messaging service for Windows NT and the enterprise.

Alan Dickman is a distributed computing architect and Ken Walker is a senior software engineer with PSW Technologies. They can be reached at alan@pswtech.com and kwalker@pswtech.com .

Back to Labs