Problem
A write operation to a TCP socket has failed on a Windows client, resulting in a “Status 24 - socket write failed” error. The backup process reports a Windows OS error due to the failure of the client network layer to send the backup data across the established link to the receiving server.
Error Message
Client bpbkar logs:
2:18:23.023 PM: [1912.2224] <2> TransporterRemote::write[2](): DBG - | An Exception of type [SocketWriteException] has occured at: | Module: @(#) $Source: src/ncf/tfi/lib/TransporterRemote.cpp,v $ $Revision: 1.51 $ , Function: TransporterRemote::write[2](), Line: 307 | Local Address: [::]:0 | Remote Address: [::]:0 | OS Error: 10054 (An existing connection was forcibly closed by the remote host.
Media server bpbrm logs:
14:18:23.075 [32163] <32> bpbrm handle_backup: from client nbclient1: FTL - socket write failed
14:18:23.130 [32163] <2> job_monitoring_exex: ACK disconnect
14:18:23.130 [32163] <2> job_disconnect: Disconnected
14:18:23.130 [32163] <4> db_error_add_to_file: from client nbclient1: FTL - socket write failed
14:18:23.179 [32163] <2> bpbrm handle_backup: client nbclient1 EXIT STATUS = 24: socket write failed
14:18:23.180 [32163] <2> inform_client_of_status: INF - Server status = 24
Cause
There are several possible causes for this issue:
- High network load
- Intermittent connectivity
- Packet reordering
- Network cable, switch port, NIC hardware issues
- Downrev NIC driver firmware
- Antivirus interfering with NetBackup process or with Windows TCP activity
Solution
Suggested steps...
- Update NIC Driver
- Open device manager on the client server.
- Go to network adapter and open backup NIC properties.
- Click on the driver tab.
- Make note of the NIC Model and Driver date.
- Download and install the latest NIC driver from NIC vendor website.
- Disable Offload Settings
- Open NIC Advanced properties Tab.
- Review the list of entries containing the word 'offload'.
- Disable all 'offload' settings. This will force the CPU to perform the task instead of added load on the NIC.
- (Optional) Disable NIC Offload/Chimney Settings using Netsh Command
- Run Windows ‘netsh’ command to disable all NIC offload/chimney settings.
- To view the setting, enter the following command: C:\> netsh int tcp show global
- To disable the offload/chimney settings, enter the following command: C:\> netsh int tcp set global chimney=disabled
- Increase Windows TCP Packet Retries
- Increase Windows TCP packet retries to a higher setting.
- The following settings will make Windows TCP more resilient for network congestion during heavy backup loads.
- Be sure to backup the registry before performing Windows Registry changes with ‘regedit’.
- Go to the following registry location: HKey_LocalMachine\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters
- On the top bar, click Edit > New > DWORD 32-bit value.
- Add the following registry values:
- Value Name: TcpMaxDataRetransmissions
- Data Type: DWORD 32-bit value
- Once created, edit it and insert value = 15 (Decimal)
- Value Name: KeepAliveTime
- Data Type: DWORD 32-bit value
- Once created, edit it and insert value = 750000 (Decimal)
- Value Name: KeepAliveInterval
- Data Type: DWORD 32-bit value
- Once created, edit it and insert value = 15000 (Decimal)
- Close the Regedit window.
- Note: The registry changes require a reboot for the Windows OS to pick up the changes.
- For more details on TCP Keepalive Best Practices - detecting network drops and preventing idle socket timeout, please see the Related Articles below.
- Configure Antivirus Exception Lists
- Antivirus applications are known to block daemons/close the established socket.
- Follow the article for proper configuration of the Antivirus exception lists.
- General recommendations for virus scanner exclusions working with NetBackup
Applies To
Operating Systems: All Windows Servers