MSMQ provides robust and easy-to-use messaging capabilities that let you send and receive messages between applications running on one machine, different machines, and even across the Internet, both synchronously and asynchronously. Unfortunately, MSMQ is notoriously underused. Find out how to get started with MSMQ and what you can do with it.
icrosoft .NET developers can use several tools to perform operations asynchronously; that is, beginning a process without waiting for the operation to complete. MSMQ has provided this facility to developers since its inception. It was especially useful for VB developers, because before MSMQ, there were very few options for managing an asynchronous process in VB. As awkward as it now seems, the most elegant solution involved using SQL Server tables to manage the 'queued' processes. Despite the multitude of tools available today for managing processes asynchronously, such as MSMQ, BizTalk, and other .NET-specific mechanisms, many developers continue to use SQL Server. This article provides an overview of MSMQ for .NET developers, including code samples for several major MSMQ features that you can use in your own applications.At its most basic level, MSMQ is a tool for sending and receiving messages. MSMQ is powerful enough to provide near real-time message exchange throughput, depending on the machines and transports involved in the exchange, and flexible enough to provide many features required in assembling a communications infrastructure for your application system. You can send MSMQ messages across machine boundaries—including across the Internet—to queues where a receiving application can retrieve them in a prioritized first-in first-out manner. MSMQ also integrates with Active Directory to provide domain-wide queue management features.
System.Messaging Namespace
This article covers MSMQ functionality exposed through the System.Messaging namespace in the .NET Framework. As such, you need to add a reference to the System.Messaging assembly before attempting to work with any of the sample code provided. The System.Messaging namespace provides access to functionality available in MSMQ 2.0, available on Windows 2000, as well as MSMQ 3.0, which is only available on Windows XP and Windows Server 2003. This article focuses the more broadly available MSMQ 2.0 features, reserving discussion of some of the features new to MSMQ 3.0 for a later article.
Workgroup Mode and Active Directory Integration
MSMQ was designed to integrate with Active Directory (AD) by publishing information about its queues in the AD directory structure, which facilitates searching for queues and testing for the presence of queues across the entire domain. However, I have found MSMQ to be much more reliable when you install it in Workgroup Mode, because that mode explicitly instructs MSMQ to disable AD integration. The main functionality that you lose in Workgroup Mode is queue search capabilities, and the ability to use public queues. Neither tradeoff is significant enough to warrant the integration with AD.
Public and Private Queues
MSMQ has two basic types of message queues: public and private. There is little functional difference between the two types, except that MSMQ publishes information about public queues in AD. AD provides many useful ways of searching for queues throughout the domain. Unfortunately, the ability to search for public queues comes at a great cost—reliance on AD considerably reduces the robustness of MSMQ (see my article titled Nine Tips to Enterprise-proof MSMQ for more information about AD integration with MSMQ). Private queues require a more absolute naming convention, and you can't search for them as easily as public queues.
Referencing Queues
To send a message to a queue, the sending application must first provide the queue location. MSMQ provides several different methods for identifying queues; the most commonly used queue references are the path name and the direct format name. You commonly use path names to reference public queues, and local private queues. Because they query AD for queue information, you can't use path names to reference private queues located on remote machines. Instead, developers commonly use direct format names to access remote private queues. Direct format names provide a means of interacting with queues directly, avoiding the need for MSMQ to query AD. This fact makes using direct format names the preferred method of accessing queues of all types, because not only does the queuing infrastructure become more robust, but message throughput increases considerably.
Both pathnames and direct format names use similar formats. The first portion of the reference specifies the location of the queue by either specifying the machine name or the network address of the server hosting the queue. The second portion of the reference is the queue name, which can include an optional attribute to specify that the queue is private. Table 1 shows several examples of queue references.
Table 1. Sample pathnames and direct format names for both public and private queues.
| Path Names | |
| .\DevXTestQueue | References a public queue named DevXTestQueue on the local machine. |
| server01\DevXTestQueue | References a public queue named DevXTestQueue on a machine named server01 (local queues can be referenced with a machine name as well). |
| .\private$\DevXTestQueue | References a private queue named DevXTestQueue on the local machine. |
| server01\private$\DevXTestQueue | References a private queue named DevXTestQueue on a machine named server01. |
| Direct Format Names | |
| Formatname:DIRECT=OS:.\DevXTestQueue | References a public queue named DevXTestQueue on the local machine. |
| Formatname:DIRECT=OS:server01\DevXTestQueue | References a public queue named DevXTestQueue on a machine named server01. |
| Formatname:DIRECT=TCP:127.0.0.1\private$\DevXTestQueue | References a private queue named DevXTestQueue on the local machine. |
Message Formatters
MSMQ under .NET also provides the ability to serialize objects and forward them to other applications. This is really nothing unique, because developers can very easily send objects to other applications using .NET Remoting. But MSMQ provides the ability to deliver those calls asynchronously, in a way that allows the message to survive machine reboots and persist indefinitely. To transfer objects, MSMQ requires you to specify a message formatter to serialize the message. The three main serializers are the ActiveXMessageFormatter, used to serialize simple formats and ActiveX objects; the BinaryMessageFormatter, used to format objects using binary serialization; and the XMLMessageFormatter, which uses XML to serialize messages. Because most messages sent using MSMQ are text strings, you'll probably use the XMLMessageFormatter most often. However, for sending and receiving objects, the binary formatter provides faster messaging with a smaller memory footprint. Formatters are very simple to use; you simply specify which formatter to use when sending a message and which one to use when receiving a message. You'll see examples of how to apply formatters in the downloadable code samples.
Developer-friendly Application Features
MSMQ provides several common features that you'll use in most projects. These features have been part of MSMQ from the beginning; without them, MSMQ developers would probably need to build these features into a "wrapper" or management application to provide the same functionality.
The CorrelationID Property
You use the CorrelationId property of a message to correlate two or more messages with one another. The typical usage scenario involves a client sending a request to a server for processing, and the server sending a message to the client when processing is complete. If this process is managed asynchronously, you will need a way to correlate responses with their respective requests—enter the CorrelationId property. In the scenario above, the client would retrieve the MSMQ-assigned ID property of the message being sent, and store it in an application-specific state table. On the server side, as the application creates response messages, it can set the CorrelationId of the response message to the value of the ID property of the request message. When the client application receives response messages from the server, it compares the CorrelationId property of the response message to the value of the message ID it had stored previously. The primary value of the CorrelationId is in correlating requests and responses.
In the code sample below, the client stores the original ID of the request message. The server code uses the request message's ID property to set the CorrelationId property of the response message. Then, when the client receives the response message, it uses the original request message ID to match the response message CorrelationId being returned from the server process.
For example, here's a client implementation that sends a request.
Private Sub SendRequestMessage(ByVal InternalState As String)
Dim RequestQueue As System.Messaging.MessageQueue
Dim RequestMessage As System.Messaging.Message
Dim MessageId As String
' open send queue
RequestQueue = GetRequestQueue()
' create message and set properties
RequestMessage = New Message
RequestMessage.Body = GetRequest(InternalState)
MessageId = RequestMessage.Id()
' send message
RequestQueue.Send(RequestMessage)
' store message state information
StoreReqeust(InternalState, MessageId)
End Sub
Private Sub SendResponse(ByVal RequestMessage As Message, _
ByVal Response As String)
Dim ResponseMessage As New System.Messaging.Message(Response)
Dim ResponseQueue As System.Messaging.MessageQueue
ResponseQueue = GetResponseQueue()
' set the response message correlationId
ResponseMessage.CorrelationId = RequestMessage.Id
ResponseMessage.Body = Response
' send the response message
ResponseQueue.Send(ResponseMessage)
End Sub
Private Sub NewMessageArrived(ByVal ResponseMessage As Message)
Dim InternalState As String
InternalState = LookupRequest(ResponseMessage.CorrelationId)
ProcessResponse(InternalState, ResponseMessage)
End Sub
The ResponseQueue Property
MSMQ provides an easy mechanism to correlate requests with responses, but—assuming this is a fault-tolerant solution running on multiple machines—how does the server know where to send response messages? You could make the response location a property of the message payload, incorporating it into the design of the message you are sending; however, proper separation of the payload from the delivery mechanism prohibits this form of mingling. To make this easier, MSMQ provides a ResponseQueue property on the Message object that you can use to provide the response location. Building on the scenario shown in the preceding code fragments, the client application would set the ResponseQueue property of the request message to specify the queue to which response messages should be sent. When the server is ready to send the response message, it opens the queue specified in the ResponseQueue property of the request message, and sends the response message to it.
When reading through the last code sample demonstrating the use of the CorrelationId property, did you wonder how the GetResponseQueue() function might be implemented? The following code sample shows how to implement the same code using the ResponseQueue property. It's an evolution of the previous code sample, demonstrating the client setting the ResponseQueue property of the request message, and the server using that ResponseQueue property as the destination queue for the response message. Note that this doesn't affect client processing of the response message.
Here's the client implementation for sending a request.
Private Sub SendRequestMessage(ByVal InternalState As String)
Dim RequestQueue As System.Messaging.MessageQueue
Dim ResponseQueue As System.Messaging.MessageQueue
Dim RequestMessage As System.Messaging.Message
Dim MessageId As String
' open request and response queues
RequestQueue = GetRequestQueue()
ResponseQueue = New MessageQueue(".\DevXResponseQueue")
' create message and set properties
RequestMessage = New Message
RequestMessage.Body = GetRequest(InternalState)
RequestMessage.ResponseQueue = ResponseQueue
MessageId = RequestMessage.Id()
' send message
RequestQueue.Send(RequestMessage)
' store message state information
StoreReqeust(InternalState, MessageId)
End Sub
Private Sub SendResponse(ByVal RequestMessage As Message, _
ByVal Response As String)
Dim ResponseMessage As New System.Messaging.Message(Response)
Dim ResponseQueue As System.Messaging.MessageQueue
' the response queue is retrieved from the request message
ResponseQueue = RequestMessage.ResponseQueue
' set message properties
ResponseMessage.CorrelationId = RequestMessage.Id
ResponseMessage.Body = Response
' send the response message
ResponseQueue.Send(ResponseMessage)
End Sub
Private Sub NewMessageArrived(ByVal ResponseMessage As Message)
Dim InternalState As String
InternalState = LookupRequest(ResponseMessage.CorrelationId)
ProcessResponse(InternalState, ResponseMessage)
End Sub
In some instances, request messages are valid only for a limited time, when delayed processing of the messages could be detrimental, or at least superfluous. In a reservations system for example, requests for reservations after the actual reservation time would be unnecessary. Fortunately, MSMQ provides a facility for handling time-sensitive messages. By setting a time period for either the message to reach the destination queue (TimeToReachQueue) or for the message to be retrieved from the queue by the receiving application (TimeToBeReceived), you can be sure that messages will never be processed later than you intend.
When a message in a queue reaches the end of one of the two timeout periods, MSMQ either discards the message or moves it to the machine's dead-letter queue, which is an administrative queue created automatically by MSMQ. By default, MSMQ discards messages when they timeout, but you can force MSMQ to save the messages in the dead-letter queue on the machine where the message's timeout period expired by setting the UseDeadLetter property of the message to true.
While both properties can be set, in practice, there is probably little need to set both properties for a message. The TimeToReachQueue is useful when working with disconnected clients. It could be used to ensure that if the local machine cannot connect to the network, and hence cannot reach the remote destination queue, then the message should be abandoned. The TimeToBeReceived is probably more useful in that it encompasses both times (time to reach queue, and time to be retrieved from the queue). In most cases, it is likely that the amount of time the message sits waiting to be processed, whether in an outgoing queue on the sending machine or in the destination queue on the remote machine, will determine the fate of the message. In the event that you decide both timeouts should be used, be aware that the TimeToBeReceived time span should be longer than the TimeToReachQueue time span; otherwise, the TimeToReachQueue time span would never expire.
The code below demonstrates setting the TimeToBeReceived property of the message to 1 minute. Note that no other functions of the system change to support this feature.
' Client implementation for sending the request.
Private Sub SendRequestMessage(ByVal InternalState As String)
Dim RequestQueue As System.Messaging.MessageQueue
Dim ResponseQueue As System.Messaging.MessageQueue
Dim RequestMessage As System.Messaging.Message
Dim MessageId As String
' open the request and response queues
RequestQueue = GetRequestQueue()
ResponseQueue = New MessageQueue(".\DevXResponseQueue")
' create the message and set message properties
RequestMessage = New Message
RequestMessage.Body = GetRequest(InternalState)
RequestMessage.ResponseQueue = ResponseQueue
RequestMessage.TimeToBeReceived = New TimeSpan(0, 1, 0)
MessageId = RequestMessage.Id()
' send the message
RequestQueue.Send(RequestMessage)
' store the message state details
StoreReqeust(InternalState, MessageId)
End Sub
' Server Implementation -- Processing the Reqeust
Private Sub SendResponse(ByVal RequestMessage As Message, _
ByVal Response As String)
Dim ResponseMessage As New System.Messaging.Message(Response)
Dim ResponseQueue As System.Messaging.MessageQueue
' the response queue is retrieved from the request message
ResponseQueue = RequestMessage.ResponseQueue
' set message properties
ResponseMessage.CorrelationId = RequestMessage.Id
ResponseMessage.Body = Response
' send the response message
ResponseQueue.Send(ResponseMessage)
End Sub
' Client Implementation -- Handling the Response
Private Sub NewMessageArrived(ByVal ResponseMessage As Message)
Dim InternalState As String
InternalState = LookupRequest(ResponseMessage.CorrelationId)
ProcessResponse(InternalState, ResponseMessage)
End Sub
MSMQ Administration in .NET
MSMQ provides surprising simple management of MSMQ queues. You can can create, delete, and purge queues with little effort. Further, you can set many queue properties programmatically, including permissions assignment.
Creating, Deleting, and Purging Queues
To create a queue, simply call the shared CreateQueue method, passing the pathname identifying the queue. Similarly, you can delete and purge (remove all messages) queues . See the code samples below.
' create a private queue on the local machine named DevXTestQueue
MessageQueue.Create(".\private$\DevXTestQueue")
' delete the private queue on the local machine
' named DevXTestQueue
MessageQueue.Delete(".\private$\DevXTestQueue")
' purge the private queue on the local machine
' named DevXTestQueue
MessageQueue.Purge(".\private$\DevXTestQueue")
MSMQ provides another shared method named Exists, which you can use to test for the presence of a queue. The only method parameter is the pathname of the queue. This method is quite expensive, so it is not a method you should use in your code during the normal course of message processing. The expense is incurred because, except for local private queues, the Exists method queries AD for information about the specified queue. Remember that because the method uses AD to find queues, you can't use Exists to test for the presence of remote private queues—you can run queries for private queues only on the machine where the queue resides. The code sample below demonstrates using Exists before attempting to create a queue.
' Using Exists to Test for the Presence of a Queue
Private Function CreateQueue(ByVal PathName As String) _
As MessageQueue
If MessageQueue.Exists(PathName) = True Then
Return New MessageQueue(PathName)
Else
Return MessageQueue.Create(PathName)
End If
End Function
Until .NET made the System.Messaging functionality of MSMQ available, VB developers found it difficult to set various properties of queues. One common change made to a queue after it is created is the assignment of permissions to the queue. While there are many ways to programmatically assign permissions to a queue, the most straightforward way to do this is to call the SetPermissions method on the appropriate queue object. By passing a string representing the user to whom permission should be granted (or revoked), the permission that is being affected, and an action (Set, Revoke, Allow, or Deny), you can set the permissions as intended. Unlike the Exists method, you can set permissions on both local and remote queues and on both public and private queues.
The code below demonstrates granting full control permissions (and subsequently revoking the same permission) to the user michael.jones@passporthealth.com on a queue named DevXTestQueue.
' Using SetPermissions to Grant and Revoke Permissions on a Queue
myQueue = New MessageQueue(".\private$\DevXTestQueue)
myQueue.SetPermissions("michael.jones@passporthealth.com", _
MessageQueueAccessRights.FullControl, AccessControlEntryType.Set)
myQueue = New MessageQueue(".\private$\DevXTestQueue)
myQueue.SetPermissions("michael.jones@passporthealth.com", _
MessageQueueAccessRights.FullControl, _
AccessControlEntryType.Revoke)
Tools available in System.Messaging also have several methods that can enumerate the public and private queues on a specified machine or all queues across an entire domain. Two of the most useful of these methods are GetPublicQueuesByMachine and GetPrivateQueuesByMachine. As their names suggest, you provide the name of a machine, and they both return an array of MessageQueue objects present on the specified machine. The code below demonstrates this functionality.
' Using GetPublicQueuesByMachine and GetPrivateQueuesByMachine to
' Enumerate Queues
myQueues = MessageQueue.GetPublicQueuesByMachine(".")
Debug.WriteLine("---- " & myQueues.GetLength(0).ToString & _
" Queues ----")
For Each myQueue As MessageQueue In myQueues
Debug.WriteLine(myQueue.QueueName)
Next
myQueues = MessageQueue. GetPrivateQueuesByMachine("SERVER1")
Debug.WriteLine("---- " & myQueues.GetLength(0).ToString & _
" Queues ----")
For Each myQueue As MessageQueue In myQueues
Debug.WriteLine(myQueue.QueueName)
Next
Messaging with MSMQ
The System.Messaging namespace provides a concise way to send a simple message to a queue. Simply open the queue, and send a simple string-based message. This option does not support modifying the default attributes of the message before sending it. The code below demonstrates this functionality. The Try/Catch block was added for completeness.
Private Sub SendMessage()
Dim myQueue As MessageQueue
Try
myQueue = New MessageQueue(".\private$\DevXTestQueue")
myQueue.Send("This is a sample message body")
Catch ex As Exception
MessageBox.Show("Exception was thrown: " & _
ex.Source & ": " & ex.Message)
End Try
End Sub
Private Sub SendMessage(Byval MessageBody as String)
Dim RequestQueue As System.Messaging.MessageQueue
Dim ResponseQueue As System.Messaging.MessageQueue
Dim RequestMessage As System.Messaging.Message
' open the request and receive queues
RequestQueue = GetRequestQueue()
ResponseQueue = GetResponseQueue()
' create the message and set properties
RequestMessage = New Message
RequestMessage.Body = MessageBody
RequestMessage.ResponseQueue = ResponseQueue
RequestMessage.TimeToBeReceived = New TimeSpan(0, 1, 0)
MessageId = RequestMessage.Id()
' send the message
RequestQueue.Send(RequestMessage)
End Sub
Until now, the discussion has centered on sending messages, but it's time to look at the process of receiving messages. To receive a message, you create a queue reference for the queue from which you expect to receive a message, and then call the MSMQ Receive method, specifying the amount of time to wait for a message to arrive in the queue, expressed as a TimeSpan object. The TimeSpan object has several overloaded constructors, but they are all self-explanatory. See the code sample below.
Private Sub ReceiveMessage()
Dim myQueue As MessageQueue
Dim myMsg As System.Messaging.Message
Try
' create a queue reference
myQueue = New MessageQueue(".\private$\DevXTestQueue")
' set the formatter for the received message
myQueue.Formatter = New XmlMessageFormatter( _
New Type() {GetType(System.String)})
' retrieve the message from the queue, waiting for no
' longer than X seconds
myMsg = myQueue.Receive(New TimeSpan(0, 0, 5))
' display the message
If Not IsNothing(myMsg) Then
MessageBox.Show("Message Received: " & CStr(myMsg.Body))
End If
' handle any exceptions that were raised
Catch exQueue As MessageQueueException
MessageBox.Show("The receive timeout expired " & _
"before a message was received.")
Catch ex As Exception
MessageBox.Show("Generic Exception was thrown: " & _
ex.Source & ": " & ex.Message)
End Try
End Sub
Receiving a single message is a simple process, but the Receive method shown above causes the thread to block every time it's called. Depending on how often you call the Receive method, and on how long it waits for a message to arrive, your application could be waiting (blocked) for long periods of time. This could cause serious performance problems for your application.
A better approach to retrieving messages, especially when the application 'listens' to the queue for all incoming messages, is to retrieve the messages asynchronously. To retrieve the messages asynchronously, MSMQ provides a BeginReceive method that doesn't block when called. The most efficient way to receive new-message notifications is to set a callback delegate that MSMQ can call when new messages arrive. If you would like MSMQ to begin looking for another message, you must instruct MSMQ to begin another asynchronous receive operation. In the code below, the MSMQ_ReceiveCompleted procedure receives the message and instructs MSMQ to begin another asynchronous receive.
Private Sub StartListening()
Dim myReceiveQueue As MessageQueue
Try
' open the receive queue
myReceiveQueue = New _
MessageQueue(".\private$\DevXTestQueue")
' set the receive queue formatter
myReceiveQueue.Formatter = New XmlMessageFormatter( _
New Type() {GetType(System.String)})
' add a handler for the ReceiveCompleted event and
' call BeginReceive
AddHandler myReceiveQueue.ReceiveCompleted, _
AddressOf MSMQ_ReceiveCompleted
myReceiveQueue.BeginReceive()
Catch ex As Exception
MessageBox.Show("Generic Exception was thrown: " _
& ex.Source & ": " & ex.Message)
End Try
End Sub
Public Sub MSMQ_ReceiveCompleted( _
ByVal p_source As Object, _
ByVal p_objAsyncResult As ReceiveCompletedEventArgs)
Dim myMessage As Message
Dim myMessageQueue As MessageQueue
' cast the source parm to a MessageQueue object
myMessageQueue = CType(p_source, MessageQueue)
' calling EndReceive will return the message that was received
myMessage = myReceiveQueue.EndReceive( _
p_objAsyncResult.AsyncResult)
' do something useful with the message
MessageBox.Show(CStr(myMessage.Body))
End Sub