Accessing the EVBAAdmin web page fails with error "No accelerator server(s) are available at the moment".

Problem

Accessing the EVBAAdmin web site on the Accelerator server fails. The website lists No Accelerator server(s) are available at the moment.  Also, the Enterprise Vault Event Log on the Accelerator server lists Event ID 40966, 372 and / or 364 with the errors shown in the Error section below.

Error Message

Veritas Enterprise Vault Event Log entries (one or more possible):

Source:        Accelerator Service Processor
Event ID:     40966
Task Category: None
Event Type:     Error
Description:
A program fault has raised an exception.
Exception: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 192.168.X.X:XXXX
Diagnostic: Error Code is 10060
Type: System.Net.Sockets.SocketException

V-437-40966

Source:        Accelerator Service Processor
Event ID:      372
Task Category: None
Level:         Error
Description:
APP AS - Customer ID: -1 - Error When Starting the Accelerator Service. System.Net.Sockets.SocketException: Only one usage of each socket address (protocol/network address/port) is normally permitted
   at KVS.Accelerator.Server.CommonStarter.InitRemoting()
   at KVS.Accelerator.Server.CommonStarter.Start()
   at KVS.Accelerator.Server.AcceleratorServiceProcessor.MainThread()

V-437-372

Source:        Accelerator Service Processor
Event ID:      364
Task Category: None
Level:         Error
Description:
APP AS - Customer ID: -1 - Remoting: Failed to initiliase remoting. System.Net.Sockets.SocketException: Only one usage of each socket address (protocol/network address/port) is normally permitted
   at System.Net.Sockets.Socket.DoBind(EndPoint endPointSnapshot, SocketAddress socketAddress)
   at System.Net.Sockets.Socket.Bind(EndPoint localEP)
   at System.Net.Sockets.TcpListener.Start(Int32 backlog)
   at System.Net.Sockets.TcpListener.Start()
   at System.Runtime.Remoting.Channels.ExclusiveTcpListener.Start(Boolean exclusiveAddressUse)
   at System.Runtime.Remoting.Channels.Tcp.TcpServerChannel.StartListening(Object data)
   at System.Runtime.Remoting.Channels.Tcp.TcpServerChannel.SetupChannel()
   at System.Runtime.Remoting.Channels.Tcp.TcpServerChannel..ctor(IDictionary properties, IServerChannelSinkProvider sinkProvider, IAuthorizeRemotingConnection authorizeCallback)
   at System.Runtime.Remoting.Channels.Tcp.TcpServerChannel..ctor(IDictionary properties, IServerChannelSinkProvider sinkProvider)
   at KVS.Accelerator.Server.CommonStarter.ListenForWindowsClientOnIPv6()
   at KVS.Accelerator.Server.CommonStarter.InitRemoting()

V-437-364

Sample Dtrace of the w3wp process on the CA or DA server during the access attempt:

[3956]    (w3wp)    <7376>    EV-L    {GLOBAL_ASAX.EN_US} {C--} [EVLAB\ev_svc] Request for http://localhost/evbaadmin/login.aspx
[3956]    (w3wp)    <7376>    EV-M    {GLOBAL_ASAX.EN_US} {C--} [EVLAB\ev_svc] Starting session
[3956]    (w3wp)    <7376>    EV-L    {-} {ACCELERATORCACHE.EN_US} {C--} ServerCache: BO_Install NOT cached
[3956]    (w3wp)    <7376>    EV-L    {-} {ACCELERATORCACHE.EN_US} {C--} ServerCache: object BO_Install Key='BO_Install' added to the cache with following options:|Dependency:'<None>'|AbsoluteExpiration:'<None>'|SlidingExpiration:'5'|CacheItemPriority:'Default'
[3956]    (w3wp)    <7376>    EV-L    {-} {CLIENTCHANNELSINK.EN_US} {C--} ClientChannelSink ProcessMessage !!ERROR!! Exception: URI: tcp://localhost:8085/DiscoveryInstall -- Method: GetDS -- Error No connection could be made because the target machine actively refused it 127.0.0.1:8085
[3956]    (w3wp)    <7376>    EV-L    {-} {CLIENTCHANNELSINK.EN_US} {C--} ClientChannelSink SyncProcessMessage !!ERROR!!: URI: tcp://localhost:8085/DiscoveryInstall -- Method: GetDS -- Error: No connection could be made because the target machine actively refused it

Review of the IIS Logs during the access attempt displays entries similar to:

127.0.0.1 GET /ev_svc - 80 - 127.0.0.1 Mozilla/5.0+(compatible;+MSIE+10.0;+Windows+NT+6.1;+WOW64;+Trident/6.0) 401 2 5 0

Cause

Access to the Compliance Accelerator (CA) or Discovery Accelerator (DA) EVBAAdmin web-based administration page is accomplished through TCP Port 8085 on the CA / DA server by default.  Access to a CA or DA Customer through the CA or DA Client is accomplished through TCP Port 8086.  There are many causes where this access fails with the error No Accelerator server(s) are available at the moment.  For a list of possible causes, refer to the KM Article 100001601 in the Related Articles section of this document.

One possible cause of the error:

• throws any of the above noted Event ID errors into the Veritas Enterprise Vault Event Log when attempting to access the EVBAAdmin web site.  Manually accessing some of the other pages on the EVBAAdmin web site, such as http://localhost/EVBAAdmin/banner.aspx or http://localhost/EVBAdmin/help.aspx, is successful. The issue is only seen when access is attempted to EVBAAdmin, which redirects to http://localhost/EVBAAdmin/login.aspx.
• will have a Dtrace log of the w3wp process during the access attempt to display entries similar to those noted in the Error section of this Document.
• will have an IIS Log entry for the time of the access attempt similar to that noted in the Errors section of this document.

This issue occurs when port 8085 and / or 8086 is blocked on the Accelerator server, typically due to another application using either port, so that CA and DA cannot use it. These ports are used when an authentication attempt is made, such as accessing the EVBAAdmin or a Customer website. The Accelerators make use of these 2 ports by default:

1. 8085: used to communicate inbound to the Accelerator server (listening port).
2. 8086: used to communicate outbound from the Accelerator server, typically with the Accelerator Client.

In the errors listed above, port 8085 or 8086 is being blocked, thus not allowing the authentication information to be passed back to the EVBAAdmin web site.
 

• Possible causes for TCP Port 8085 and / or 8086 blocking are, but not limited to, the following:
  • An application using ports 8085 and 8086.
  • Windows Firewall
  • Some other software firewall.
  • Firewall configured as part of an antivirus application.
  • Group Policy

Solution

The Accelerator server and/or the environment will need to be reviewed to determine the cause of the port block.

To test for an application using ports 8085 and 8086 on the Accelerator Server, refer to Solution Step 6 in the KM Article 100001601 for detailed steps.  A summary of the steps is as follows:

1. Open a Command Prompt on the Accelerator server.
2. At the prompt, type:
   1. netstat -aon | findstr 8085
3. If this produces a result, cross-reference the PID (Process Identifier) in the last column of the netstat output with the entries in Task Manager to find the application using this port. The application using this port will need to be re-configured to use another port. The Task Manager | Services tab has the PID listed. The Task Manager Processes tab will need to have the PID listed under View | Select Columns.
4. If no results are listed, re-run the netstat command for port 8086:
   1. netstat -aon | findstr 8086
5. If there are still no results, then there is no application using these ports.

To test for a blocked port 8085 and / or port 8086 on the Accelerator server:

1. Stop the Enterprise Vault Accelerator Manager Service (EVAMS).
2. Open Windows Explorer on the Accelerator Server and browse to the Accelerator install folder, typically under C:\Program Files (x86)\Enterprise Vault Business Accelerator\.
3. Edit the following files using Notepad to replace all entries for 8085 with 4085 and / or 8086 with 4086 for CA and DA unless otherwise noted:
   1. AcceleratorLegalHold.dll.config (2 entries similar to the following) [needed for DA only but also exists on CA]
      1. On or near line 25:     <add key="Remoting Channel Configuration" value="name=Client Port, port=8085,suppressChannelData=false, machineName=, priority=1, secure=true, protectionLevel=EncryptAndSign, useIpAddress=true,  bindTo=0.0.0.0, rejectRemoteRequests=false, exclusiveAddressUse=true, impersonate=false, authorizationModule=, typeFilterLevel=Full" />
      2. On or near line 26:     <add key="Remoting Channel Configuration IPv6" value="name=Client Port IPv6, port=8085,suppressChannelData=false, machineName=, priority=2, secure=true, protectionLevel=EncryptAndSign, useIpAddress=true,  bindTo=[::], rejectRemoteRequests=false, exclusiveAddressUse=true, impersonate=false, authorizationModule=, typeFilterLevel=Full" />
      3. On or near line 28:     <add key="Windows Client Remoting Channel Configuration" value="name=Windows Client Channel, port=8086,suppressChannelData=false, priority=1, secure=true, protectionLevel=EncryptAndSign, rejectRemoteRequests=false, exclusiveAddressUse=true, impersonate=false, typeFilterLevel=Full" />
      4. On or near line 29:     <add key="Windows Client Remoting Channel Configuration IPv6" value="name=Windows Client Channel IPv6, port=8086,suppressChannelData=false, priority=2, secure=true, protectionLevel=EncryptAndSign,  bindTo=[::], rejectRemoteRequests=false, exclusiveAddressUse=true, impersonate=false,  typeFilterLevel=Full" />
   2. AcceleratorManager.exe.config (2 entries similar to the following)
      1. On or near line 25:     <add key="Remoting Channel Configuration" value="name=Client Port, port=8085,suppressChannelData=false, machineName=, priority=1, secure=true, protectionLevel=EncryptAndSign, useIpAddress=true,  bindTo=0.0.0.0, rejectRemoteRequests=false, exclusiveAddressUse=true, impersonate=false, authorizationModule=, typeFilterLevel=Full" />
      2. On or near line 26:     <add key="Remoting Channel Configuration IPv6" value="name=Client Port IPv6, port=8085,suppressChannelData=false, machineName=, priority=2, secure=true, protectionLevel=EncryptAndSign, useIpAddress=true,  bindTo=[::], rejectRemoteRequests=false, exclusiveAddressUse=true, impersonate=false, authorizationModule=, typeFilterLevel=Full" />
      3. On or near line 28:     <add key="Windows Client Remoting Channel Configuration" value="name=Windows Client Channel, port=8086,suppressChannelData=false, priority=1, secure=true, protectionLevel=EncryptAndSign, rejectRemoteRequests=false, exclusiveAddressUse=true, impersonate=false, typeFilterLevel=Full" />
      4. On or near line 29:     <add key="Windows Client Remoting Channel Configuration IPv6" value="name=Windows Client Channel IPv6, port=8086,suppressChannelData=false, priority=2, secure=true, protectionLevel=EncryptAndSign,  bindTo=[::], rejectRemoteRequests=false, exclusiveAddressUse=true, impersonate=false,  typeFilterLevel=Full" />
   3. AcceleratorManagerConsole.exe.config (2 entries similar to the following)
      1. On or near line 25:     <add key="Remoting Channel Configuration" value="name=Client Port, port=8085,suppressChannelData=false, machineName=, priority=1, secure=true, protectionLevel=EncryptAndSign, useIpAddress=true,  bindTo=0.0.0.0, rejectRemoteRequests=false, exclusiveAddressUse=true, impersonate=false, authorizationModule=, typeFilterLevel=Full" />
      2. On or near line 26:     <add key="Remoting Channel Configuration IPv6" value="name=Client Port IPv6, port=8085,suppressChannelData=false, machineName=, priority=2, secure=true, protectionLevel=EncryptAndSign, useIpAddress=true,  bindTo=[::], rejectRemoteRequests=false, exclusiveAddressUse=true, impersonate=false, authorizationModule=, typeFilterLevel=Full" />
      3. On or near line 28:     <add key="Windows Client Remoting Channel Configuration" value="name=Windows Client Channel, port=8086,suppressChannelData=false, priority=1, secure=true, protectionLevel=EncryptAndSign, rejectRemoteRequests=false, exclusiveAddressUse=true, impersonate=false, typeFilterLevel=Full" />
      4. On or near line 29:     <add key="Windows Client Remoting Channel Configuration IPv6" value="name=Windows Client Channel IPv6, port=8086,suppressChannelData=false, priority=2, secure=true, protectionLevel=EncryptAndSign,  bindTo=[::], rejectRemoteRequests=false, exclusiveAddressUse=true, impersonate=false,  typeFilterLevel=Full" />
   4. AcceleratorService. Exe.config (2 entries similar to the following)
      1. On or near line 25:     <add key="Remoting Channel Configuration" value="name=Client Port, port=8085,suppressChannelData=false, machineName=, priority=1, secure=true, protectionLevel=EncryptAndSign, useIpAddress=true,  bindTo=0.0.0.0, rejectRemoteRequests=false, exclusiveAddressUse=true, impersonate=false, authorizationModule=, typeFilterLevel=Full" />
      2. On or near line 26:     <add key="Remoting Channel Configuration IPv6" value="name=Client Port IPv6, port=8085,suppressChannelData=false, machineName=, priority=2, secure=true, protectionLevel=EncryptAndSign, useIpAddress=true,  bindTo=[::], rejectRemoteRequests=false, exclusiveAddressUse=true, impersonate=false, authorizationModule=, typeFilterLevel=Full" />
      3. On or near line 28:     <add key="Windows Client Remoting Channel Configuration" value="name=Windows Client Channel, port=8086,suppressChannelData=false, priority=1, secure=true, protectionLevel=EncryptAndSign, rejectRemoteRequests=false, exclusiveAddressUse=true, impersonate=false, typeFilterLevel=Full" />
      4. On or near line 29:     <add key="Windows Client Remoting Channel Configuration IPv6" value="name=Windows Client Channel IPv6, port=8086,suppressChannelData=false, priority=2, secure=true, protectionLevel=EncryptAndSign,  bindTo=[::], rejectRemoteRequests=false, exclusiveAddressUse=true, impersonate=false,  typeFilterLevel=Full" />
   5. ADSynchroniser.exe.config (2 entries similar to the following)
      1. On or near line 25:     <add key="Remoting Channel Configuration" value="name=Client Port, port=8085,suppressChannelData=false, machineName=, priority=1, secure=true, protectionLevel=EncryptAndSign, useIpAddress=true,  bindTo=0.0.0.0, rejectRemoteRequests=false, exclusiveAddressUse=true, impersonate=false, authorizationModule=, typeFilterLevel=Full" />
      2. On or near line 26:     <add key="Remoting Channel Configuration IPv6" value="name=Client Port IPv6, port=8085,suppressChannelData=false, machineName=, priority=2, secure=true, protectionLevel=EncryptAndSign, useIpAddress=true,  bindTo=[::], rejectRemoteRequests=false, exclusiveAddressUse=true, impersonate=false, authorizationModule=, typeFilterLevel=Full" />
      3. On or near line 28:     <add key="Windows Client Remoting Channel Configuration" value="name=Windows Client Channel, port=8086,suppressChannelData=false, priority=1, secure=true, protectionLevel=EncryptAndSign, rejectRemoteRequests=false, exclusiveAddressUse=true, impersonate=false, typeFilterLevel=Full" />
      4. On or near line 29:     <add key="Windows Client Remoting Channel Configuration IPv6" value="name=Windows Client Channel IPv6, port=8086,suppressChannelData=false, priority=2, secure=true, protectionLevel=EncryptAndSign,  bindTo=[::], rejectRemoteRequests=false, exclusiveAddressUse=true, impersonate=false,  typeFilterLevel=Full" />
   6. ImportExport.exe.config (1 entry similar to the following)
      1. On or near line 10 :     <add key="RemotePort" value="8085" />
      2. On or near line 15 :     <add key="Windows Client Remoting Channel Configuration" value="name=Windows Client Channel, port=8086,suppressChannelData=false, priority=1, secure=true, protectionLevel=EncryptAndSign, rejectRemoteRequests=false, exclusiveAddressUse=true, impersonate=false, typeFilterLevel=Full" />
4. Change Windows Explorer on the Accelerator Server to browse to the AcceleratorAdminWeb folder, typically under C:\Program Files (x86)\Enterprise Vault Business Accelerator\AcceleratorAdminWeb\.
   1. Edit the Web.config file using Notepad to replace 8085 with 4085 (1 entry similar to the following)
      1. On or near line 17 :      <add key="RemotePort" value="8085" />
5. Change Windows Explorer on the Discovery Accelerator Server to browse to the CustodianManagerWeb folder, typically under C:\Program Files (x86)\Enterprise Vault Business Accelerator\CustodianManagerWeb\.
   1. Edit the Web.config file using Notepad to replace 8085 with 4085 (1 entry similar to the following)
      1. On or near line 17 :      <add key="RemotePort" value="8085" />
6. Change Windows Explorer on the Discovery Accelerator Server to browse to the DiscoveryWeb folder, typically under C:\Program Files (x86)\Enterprise Vault Business Accelerator\DiscoveryWeb\.
   1. Edit the Web.config file using Notepad to replace 8085 with 4085 (1 entry similar to the following)
      1. On or near line 17 :      <add key="RemotePort" value="8085" />
7. Change Windows Explorer on the Compliance Accelerator Server to browse to the ComplianceWeb folder if present, typically under C:\Program Files (x86)\Enterprise Vault Business Accelerator\ComplianceWeb\.
   1. Edit the Web.config file using Notepad to replace 8085 with 4085 (1 entry similar to the following)
      1. On or near line 17 :      <add key="RemotePort" value="8085" />
8. From a Command Prompt on the Accelerator server, run the command iisreset.
9. Start the EVAMS.
10. Using a supported Web Browser, open the EVBAAdmin web site at https://localhost/EVBAAdmin.
11. If the EVBAAdmin website can be successfully opened, then port 8085 and / or port 8086 are blocked.

Port 4085 was used for this test as it falls within the well-known block of ports, typically left unblocked by most firewall and antivirus applications. Repeat the above steps to revert the files modified above to their original states by replacing 4085 with 8085 and 4086 with 8086. The environment can now be reviewed for any firewalls and/or applications that could block ports 8085 and 8086.

If no firewalls are found to be blocking either or both ports 8085 and 8086, then an application other than CA / DA will be blocking those ports by using them for itself.  Either that application must be reconfigured to use another port or be removed or CA / DA will need to be reconfigured to use other ports.  Follow the editing steps above to change the files noted to ports determined to be open through firewalls and available for use by CA / DA.

IMPORTANT:

• If port 8086 must be changed to another port, all CA / DA clients will need to be reconfigured to use the same port.  See KM Article 100002435 for instructions to reconfigure the CA / DA Client to use a port other than 8086.
• If port 8086 must be changed to another port and Enhanced Reporting through OData is to be used, the OData configuration must be changed to use the same port.  See KM Article 100044862 for instructions to reconfigure OData to use a port other than 8086.