ince the release of the first version of MSMQ in December of 1997, it has provided the plumbing required by many enterprise applications. Before MSMQ, developers implemented tools with similar functionality using input/output directories with files, SQL Server tables, DCOM, TCP/IP sockets, etc. The major shortcoming in all these approaches was the complexity of delivering a robust system?performance took a back seat. MSMQ’s release provided the standardized messaging platform that many projects needed?a great head-start for developers building enterprise solutions.

Unfortunately, MSMQ fell short in delivering the level of performance and robustness required by many projects. As message volume grew, and demand for ‘always on’ reliability increased, many found MSMQ falling short. However, after many troubleshooting sessions involving Microsoft engineers, important ways of increasing both the performance and robustness of MSMQ emerged. In this article, I’ve listed these options, arranged as tips that significantly impact messaging performance and tips that improve reliability using MSMQ.

Increasing MSMQ Performance
If you’re not already intimately familiar with MSMQ, I encourage you to review the documentation for MSMQ versions 2.0 and 3.0. You can find documentation links in the Related Resources section of this article that outline all the major features. The tips you’ll find here don’t rehash the documentation; instead, they provide advice widely applicable to most MSMQ installations.

Tip 1: Use Recoverable Messages Only When Necessary
By default, MSMQ processes messages as quickly as possible, at the risk of possibly losing the messages if the machine is rebooted or if the MSMQ service is restarted. MSMQ stores all the messages in memory to increase performance. Unless your application is designed to survive the loss of messages, you really don’t have many options?you probably need to make your messages recoverable.

This recoverability comes at a cost. In my unofficial tests, I found that messages took about 3-6 times longer to send when the messages were configured as recoverable. The real difference will depend heavily on the disks you are using on the receiving machine (and on the sending machine if the sender is disconnected from the receiver), but you get the general idea. Below is a listing of the change necessary to configure a message to be recoverable.

   Dim msmqMessage As System.Messaging.Message   msmqMessage = New System.Messaging.Message("Hello World")   msmqMessage.Recoverable = True 'default is False

The code required to retrieve recoverable messages is the same as that required to retrieve non-recoverable messages. The receiving application does not treat the messages differently. See the code sample below for an example of retrieving messages.

   Dim msmqQueue As System.Messaging.MessageQueue   Dim msmqMessage As System.Messaging.Message   msmqQueue = New MessageQueue( _      "FormatName=OS:testboxTestQueue")   msmqMessage = msmqQueue.Receive()

Recoverable messages, while they significantly decrease performance, can increase the robustness of the application considerably. If it is important that messages are not lost, then configuring for recoverable messages will help to prevent communications failures.

Tip 2: Do Not Use Journaling

Figure 1. Enabling Journaling: Use the property window of a message queue to enable journaling.

Journaling is a feature that allows a copy of the retrieved message to be copied into a special queue when it is retrieved by the receiving application. The journal messages are stored in a Journal queue associated with the queue the message was retrieved from. This feature is useful mainly for debugging and auditing, and should not be used on production systems except for brief periods. While the performance overhead of retrieving messages from a queue that is configured for Journaling is only about 20% more than retrieving messages without Journaling, the real cost is unexpected problems caused when an unchecked MSMQ service runs out of memory or the machine is out of disk space. There is no measurable difference in sending messages to a queue configured for Journaling; the overhead is incurred when retrieving messages. You can enable journaling on a queue by opening the queue properties and clicking the Enabled checkbox in the Journaling section of the window (see Figure 1).

After retrieving messages from a queue enabled for Journaling, you can view the messages in the Journal queue using the MSMQ administrative tool in Computer Management (see Figure 2). The tool shows the messages that have been retrieved from the queue. To see the details of an individual message, double-click a message in the queue’s journal. The tool lets you view the body of the message as well as details about when the message was sent and received, (see Figure 3).

Figure 2. Viewing Journaling Queue Contents: You can view the contents of a queue’s journal using the MSMQ administrative tool in Computer Management.

Figure 3. Viewing Individual Journal Messages: The properties window for a queue journal message displays the body of the message.

Tip 3: Avoid Transactions If Possible
MSMQ supports transactions for environments sensitive to ATOMIC operations. To improve performance when using transactions, MSMQ supports internal transactions?either all messages are sent or none are sent; however, to make the product more extensible, MSMQ also supports COM+-style DTC transactions, known as extexternalernal transactions. This feature allows developers to enlist the sending or receiving of a message in a transaction that involves committing other transactions such as database updates. Unless this is a feature that is absolutely required by your application, steer clear of transactions involving MSMQ. The overhead involved in transaction support for internal transactions is close to 100%. Using DTC-style transactions incur an even higher overhead typical of two-phase commit operations.

MSMQ supports internal transactions using the MessageQueueTransaction object. This object coordinates of message sends and receives inside a transaction boundary. It also exposes methods used for both controlling the outcome of a transaction and monitoring the state of existing transactions. The following code shows an example of sending two messages in the context of an internal MSMQ transaction.

   Imports System.Messaging      Dim msmqTransaction As New MessageQueueTransaction()   Dim msmqQueue As New MessageQueue( _       "FormatName=OS:testboxTestQueue")   Dim msmqMessage1 as New Message   Dim msmqMessage2 as New Message   msmqTransaction.Begin()   Try      msmqQueue.Send(msmqMessage1, msmqTransaction)      msmqQueue.Send(msmqMessage2, msmqTransaction)      msmqTransaction.Commit()   Catch      msmqTransaction.Abort()   Finally      msmqQueue.Close()   End Try   

As you can see in the previous code example, MSMQ uses imperative support for internal transactions?that is, you explicitly request transaction services. To support external transactions, you implement a .NET serviced component by inheriting from the System.EnterpriseServices.ServicedComponent class and by setting several .NET Framework attributes that declare support for transactions. External transactions use a declarative model.

The following MSMQSampleApp class shows how to implement declarative external transactions

   Imports System.EnterpriseServices                    Public Class MSMQSampleApp      Inherits ServicedComponent         Public Function SendShipperNotification() as Boolean            Try            Me.SendMSMQNotification(strCompany, strMessage)            Me.WriteSQLAuditEntry(strCompany, strMessage)            ContextUtil.SetComplete()            Return True;         Catch e As Exception            ContextUtil.SetAbort()            Return False;         End Try         End Function      End Class

This class first declares support for transactions using the appropriate .NET Framework attributes. It then defines a method that both sends an MSMQ message and writes an entry to a database inside the context of an external transaction. As you might expect, either both actions occur, or neither occurs. Unlike internal transactions, where MSMQ brokers the transaction, Microsoft’s DTC acts as the transaction broker for external transactions. The example SendShipperNotification function sends a notification message to a shipping company using MSMQ. To maintain consistency the application also writes an audit log entry to a SQL database via the WriteSQLAuditEntry method.

Tip 4: Use Windows Security Sparingly
MSMQ uses the standard Windows security model. You can configure queues to permit only senders and receivers with appropriate security privileges. Security is an ever-increasing issue at most companies. I have found that the tradeoff for including Windows security in messaging is that it takes about 2.5 times as long to send the same messages. By configuring MSMQ not to send the security descriptors associated with the sending application, you can obtain significant performance gains. To configure MSMQ not to send a security descriptor with the message, set the message property AttachSenderID to False (the default is True), for example:

   msmqMessage.AttachSenderId = False

Tip 5: Use Private Queues
MSMQ supports both public and private queues. The primary difference is that public queues, and their properties, are published in Active Directory. This makes it easier to find queues based on their name or properties anywhere in an enterprise. I have found this feature to be of little use, since I generally know which queue I am going to send a message to.

I encourage you to use private queues whenever possible. Not only are they slightly more efficient, but they are also less prone to problems with AD (no one has problems with AD do they?). When AD is misbehaving, or when the machine hosting your queues cannot reach the MSMQ service running on one of your AD servers, the public queues hosted on the machine cannot be configured and no queue properties can be obtained. You will be unable to create new queues, delete existing queues, or modify the queues until the issue is resolved.

Tip 6: Use Direct Format Names to Open Queues
When opening queues, MSMQ provides several options for specifying the queue you would like to open. The easiest way to specify a queue name is to use a simple pathname such as testmachine estqueue. This naming convention specifies that you would like to open a queue named testqueue hosted on a machine named testmachine. When opening a queue in this manner, MSMQ consults the directory service to resolve the location and properties of the queue.

By using direct format names with private queues, you specify the exact queue location and name, circumventing the overhead involved in consulting the directory service. An example of a direct format name is TCP:192.168.100.1TCP:192.168.100.155private$TestQueue. There are other ways of specifying a direct format name, for example, you can specify a machine name as in OS:textboxprivate$TestQueue, rather than an IP address. Both direct format names are equivalent, except that the latter incurs the slight overhead of looking up the machine’s IP address. To obtain the most reliable and highest performing message queuing environment, use direct format names and private queues whenever possible. This MSDN article provides other examples of format names.

Increasing Reliability
The preceding section discussed MSMQ availability only in cases where higher availability is a factor in increasing performance. This section describes several other ways of making an MSMQ messaging platform more reliable. The following tips don’t involve changes to code, but are very useful ways of making MSMQ a more reliable messaging platform.

Tip 7: Avoid Active Directory and Public Queues
When using MSMQ with public queues, the local MSMQ service communicates with the MSMQ directory service running on one of the AD domain controllers?the directory service catalogs settings for all the public queues in the domain. In my experience, there are countless problems with the directory services becoming inoperable?and that prevents access to the public queues (You just have to hope your applications aren’t using the queues when this happens). In contrast, private queues do not interact with the domain-wide directory service; only the local MSMQ service knows about the private queue. As such, you must reference private queues using direct format names.

Another problem with when using MSMQ with AD integration is that the local MSMQ service cannot start when it is unable to reach one of the domain-wide directory services. This is true even if you aren’t using any public queues. To fix this problem, elect to install MSMQ without Active Directory integration during the install process of MSMQ. If you are trying to increase the availability of your MSMQ installations, I generally recommend using direct format names with private queues after installing MSMQ without Active Directory integration. You will be thankful you chose this route.

Unfortunately, my experience has been that even when you install MSMQ without Active Directory integration, as recommended above, public queues and AD integration functionality returns. While Microsoft Support has not confirmed this behavior, I have witnessed it on several occasions. A recent call to Microsoft revealed a registry key apparently specifically intended to prevent issues of this nature?it prevents MSMQ from using AD integration. To add this value to your registry, add a DWORD entry named AlwaysWithoutDS under the subkey: HKEY_LOCAL_MACHINHKEY_LOCAL_MACHINESOFTWAREMicrosoftMSMQParametersSetup. Set the value to 1. Adding the key should prevent MSMQ from ever attempting to use AD’s Directory Services functionality and will lead to a more stable MSMQ installation.

Tip 8: Clustering and Load-balancing MSMQ
Using Microsoft’s Windows Clustering tool, queues will failover from one machine to another if one of the queue server machines stops functioning normally. The failover process moves the queue and its contents from the failed machine to the backup machine. Microsoft’s clustering works, but in my experience, it is difficult to configure correctly and malfunctions often. In addition, to run Microsoft’s Cluster Server you must also run Windows Server Enterprise Edition?a costly operating system to license. Together, these problems warrant searching for a replacement.

One alternative to using Microsoft’s Cluster Server is to use a third-party IP load-balancing solution, of which several are commercially available. These devices attach to your network like a standard network switch, and once configured, load balance IP sessions among the configured devices. To load-balance MSMQ, you simply need to setup a virtual IP address on the load-balancing device and configure it to load balance port 1801. To connect to an MSMQ queue, sending applications specify the virtual IP address hosted by the load-balancing device, which then distributes the load efficiently across the configured machines hosting the receiving applications. Not only does this increase the capacity of the messages you can process (by letting you just add more machines to the server farm) but it also protects you from downtime events caused by failed servers.

To use a hardware load balancer, you need to create identical queues on each of the servers configured to be used in load balancing, letting the load balancer connect the sending application to any one of the machines in the group. To add an additional layer of robustness, you can also configure all of the receiving applications to monitor the queues of all the other machines in the group, which helps prevent problems when one or more machines is unavailable. The cost for such queue-monitoring on remote machines is high (it’s almost always more efficient to read messages from a local queue) but the additional level of availability may be worth the cost.

Tip 9: Set Message Store Quotas

Figure 4. Setting Message Store Quotas: Windows 2000 Server instructions describe how to enable a machine quota to limit the size of the MSMQ messages store.

MSMQ has limitations is limited in on the number and size of the messages it can store on a machine. In most instances, MSMQ versions 1.0 and 2.0 can store no more than about 1.6 GB of messages on a machine. There are ways of overcoming this size limitation, but the your main goal should be to protect MSMQ from problems that arise when the message store size exceeds the amount of space allowed on the machine. When that happens, using a standard MSMQ installation, the MSMQ service will become unresponsive and, if stopped, will not start.

Using machine quotas, you can set a limitation on the size of the message store (see Figure 4 for instructions on configuring a quota for this purpose). Setting such a limit prevents The limitation will prevent MSMQ from receiving any messages until adequate storage space becomes available?the messages queue up on the sending machine. While this does not solve the problem in a way that makes your application more reliable, it helps makes MSMQ more stable in such circumstances. With MSMQ still running, you can purge queues or retrieve messages from queues on the machine until the message store has returned to a manageable size.

MSMQ is an amazingly flexible tool that can provide reliable message queuing, but like any other tool, proper configuration is essential in reaching acceptable performance and availability goals. The suggestions in this article cover only a fraction of the tunable characteristics of MSMQ. If you begin using MSMQ at an enterprise level, I encourage you to browse the additional resources listed in the Related Resources section of this article.