When a service group containing a MSMQ (Microsoft Message Queuing) comes online it may fail to bind to the registered TCP port 1801 for MSMQ. The resource will still online successfully.
This issue may manifest itself by the failure of MSMQ (which may be part of a clustered Enterprise Vault service group or any other service group containing MSMQ) to send messages to the clustered MSMQ. If the clustered MSMQ is not able to bind to the port correctly, then remote MSMQ will not be able to create the necessary outbound connector to it that will allow message flow between the two.
ERROR 2170(0xc000087a) <MSMQ$resource_name> <server>
Message Queuing failed to bind to port 1801. The port may already be bound to another process. Make sure that the port is free and try to start Message Queuing again. If this problem arises during setup, you must free the port and run setup again.
There have also been cases where the following error will also be seen in the Enterprise Vault event logs:
Log Name: Symantec Enterprise Vault
Source: Enterprise Vault
Date: 7/17/2012 10:13:50 AM
Event ID: 3425
Task Category: Storage File Watch
Error occurred when sending a MQ message.
Message Queue: DIRECT=OS:ev02.domain.com\private$\enterprise vault exchange mailbox task for exmb1 120120151154 a1
Message Label: Post Process Archived Item
Message ID: 0
Message Body size: 758
There are two causes for this issue:
An issue has been identified in the versions of SFW HA running on the Microsoft Windows operating system versions listed here where if a MSMQ driver is already listening on port 1801 when the clustered MSMQ comes online it is unable to bind also to port 1801. To confirm this is the case, perform the following steps below in the Solution section.
Message Queue location has exceeded the size limits
For Scenario 1, perform the following steps:
1. Determine all existing MSMQ port bindings
netstat -abno | findstr 1801
2. From the output it will be shown all instances of the message queue driver listening on port 1801. To see what is the instance of the driver, execute:
tasklist /svc | findstr <PID>
where PID (process ID) is listed in the output of step 1.
The output of this command should show that the clustered MSMQ driver is listening on port 1801. The output here shows a successful check:
C:\>netstat -abno |findstr 1801
TCP 10.251.59.54:1801 0.0.0.0:0 LISTENING 4148
TCP 10.251.59.118:1801 0.0.0.0:0 LISTENING 102556
TCP 10.251.59.117:1801 0.0.0.0:0 LISTENING 75272
It is known that the EV cluster group IP address is 10.251.59.117, the corresponding Process ID is 75272
C:\>tasklist /svc | findstr 75272
mqsvc.exe 75272 MSMQ$EVSG-MSMQ
This confirms the clustered MSMQ driver is bound correctly to the port. If the clustered MSMQ is not listed then it is possible that this issue applies. If the MSMQ is listed proceed to Scenario 2 below.
Note: This issue is resolved in Veritas Storage Foundation HA for Windows 5.1 SP2 CP 11 and a fix is available from Symantec Enterprise Support via Cumulative Patch 11 (CP). If you should encounter this issue, please contact Symantec Enterprise Technical Support to obtain the hotfix. The following registry keys must be created after the patch install.
HKLM\Software\Veritas\VCS\BundledAgents\Lanman\DNSLookupRetryCount = 36
HKLM\Software\Veritas\VCS\BundledAgents\Lanman\SkipDNSCheckFailure = 0
HKLM\Software\Veritas\VCS\BundledAgents\MSMQ\VirtualIPPortCheckRetryCount = 40
For Scenario 2, perform the following steps below:
1. Open the Computer Management to view the Message Queues
2. Right-click on Message Queuing | Properties
3. The General tab shows the Storage Limits. 8 gig is the recommended for Limit message storage to (KB)
4. Confirm that the storage limit has not been exceeded. Select the Storage tab and browse to the location specified.
5. If the limit has been exceeded, simply increase the size to the desired level.
6. When changing the size, it will prompt to restart the Message Queuing service. This will cause the Message Queuing Resource to fault. Simply clear the fault after the change.
Prior to making the change, it is best to Offline the EV Storage and Task Controller Resources (they depend on Message Queuing)
- Microsoft Windows Enterprise Server 2008 R2 or later
- Veritas Storage Foundation HA 5.1 SP1 or later
- Enterprise Vault 10.0