During the backup of a remote machine the error message "Connection to target system has been lost" is received for random byte counts

Problem

During the backup of a remote machine the error message "Connection to target system has been lost" is received for random byte counts.

Error Message

The Job Log would display the following errors:

Final error: 0xe000fe30 - A communications failure has occurred

Backup- \\ServerName.local\C:
V-79-57344-65072 - The connection to target system has been lost. Backup set canceled.

Verify- \\ServerName.local\C:
V-79-57344-34004 - End of data set was unexpectedly encountered.

In the debug log on the media server you get:

[5600] [ndmp\tpfmt] - Socket error mvrRet = 10054
[3248] [ndmp\tpfmt] - TF_GetNextTapeRequest( ) Error = E00084F9
[3248] [ndmp\tpfmt] - TF_GetNextTapeRequest( ): Recalled in Error State
[3248] [ndmp\tpfmt] - TF_GetNextTapeRequest( ): Recalled in Error State
[3248] [ndmp\tpfmt] - TF_CloseSet()
[3248] [ndmp\tpfmt] - Fatal error in channel. 

UMI:- V-79-57344-65072.

Cause

Winsock errors 10053 and 10054 are TCP/IP errors that occur around the network layer or below of the Open Systems Interconnection model (OSI model). On both the client and server sides, applications such as Backup Exec, Exchange, SharePoint, etc. exist at the application layer above the network layer. The responsibility for establishing and maintaining a reliable network connection belongs to the layers below Backup Exec. If stable TCP/IP connections cannot be maintained, then Backup Exec will be unable to continue the backup once these errors (10053 or 10054) are received.

In situations where this error is occurs consistently, it is often the case that copying a large file from the client machine to the media server can fail as well. This shows that the issue is at the operating system (OS) or hardware level.

Solution

1. To make sure the existing issue is occurring turn on debugging on the media server and rerun the backup.  Once the backup fails search on the debug log on the media server for: Socket error mvrRet = 10054 which will confirm that this is a network issue that is occurring during the backup.
2. Confirm that the Network Interface Card (NIC) drivers and firmware are at the latest update level.
3. Turn off RSS, NetDMA and TCP Chimney or offload on the backup target (instructions here)
4. Confirm that you can copy data over the network to the media server of at least or greater than the amount of data that is being backed up.
5. Perform Network Diagnostics using Veritas Quick Assist (VQA)