1. Support
  2. Knowledge base
  3. 100005623

VMware vStorage backups failing with system call failed(11) during periods of high activity

Article: 100005623
Last Published: 2015-09-21
Ratings: 0 1
Product(s): NetBackup

Problem

A FlashBackup of multiple vmdk's during high activity will fail:

3/10/2011 7:03:44 PM - estimated 35636842 Kbytes needed
3/10/2011 7:03:45 PM - started process bpbrm (3636)
3/10/2011 7:03:46 PM - connecting
3/10/2011 7:03:46 PM - connected; connect time: 00:00:00
3/10/2011 7:04:45 PM - begin writing
3/10/2011 8:33:38 PM - Error bpbrm(pid=3636) from client clientname: ERR - Read of VM file read 0 bytes, should have read 113664 bytes
3/10/2011 8:33:48 PM - Error bpbrm(pid=3636) could not send server status message      
3/10/2011 8:33:49 PM - end writing; write time: 01:29:04
system call failed(11)

Error Message

ERR - Read of VM file read 0 bytes, should have read xxxx bytes
system call failed(11)

 

Set up VxMS logging on the VMware Backup Host by doing the following:
Start à Run regedit. Navigate in the registry editor to:

HKEY_LOCAL_MACHINE\SOFTWARE\VERITAS\VxMS

Change the Logging DWORD value to 0x7580
The next time VxMS is run, it will create a log (ex: VxMS-1832-MMDDYY.log).  
The log is in \Program Files\Common Files\VERITAS\VxMS\Logs\.  The Logs directory will be created automatically if it does not exist.
With the above logging in place, run a backup of one of the clients that is failing.
Check the timestamp of the error in the bpbkar log. This occurred at the same time in the VxMS logs:

 [TID 0x00002248]: VixCoordinator::vixMapRead:.\VixCoordinator.cpp:1090 <VFM_ESINFO> : Obj: [WWW-01] <clientname>.vmdk
[TID 0x00002248]: g_vdInterfaceLogger:.\libvix.cpp:1693 <VFM_ESWARN> : [NFC ERROR] NfcNetTcpRead: bRead: -1
[TID 0x00002248]: g_vdInterfaceLogger:.\libvix.cpp:1693 <VFM_ESWARN> : [NFC ERROR] NfcFssrvrRecv: failed with code = 8
[TID 0x00002248]: g_vdInterfaceLogger:.\libvix.cpp:1693 <VFM_ESWARN> : [NFC ERROR] NfcFssrvr_DiskIo: failed to receive io reply
[TID 0x00002248]: g_vdInterfaceLogger:.\libvix.cpp:1689 <VFM_ESINFO> : DISKLIB-LIB   : RWv failed ioId: #49854 (290) (34) .
[TID 0x00002248]: VixFile::read:h:\701\src\vxms\plugin\common\map\vix\VixFile.h:190 <VFM_ESERROR> [Sys Error: No error] : g_vdInterface->vdRead returned error 1, offset 0x0000000000000000, length 0x000000000001BC00
[TID 0x00002248]: VixCoordinator::vixMapRead:.\VixCoordinator.cpp:1118 <VFM_ESERROR> [Sys Error: No error] : Exited with 11
[TID 0x00002248]: vix_map_read:.\libvix.cpp:1434 <VFM_ESINFO> : Error 11

Cause

It is a network error between the ESX and VMWare Backup Hosts. The NFC connection seems to have failed in mid transfer. It appears the Network Base connection cannot handle the work.

Solution

Options are:

  1. Try backing up the problem client by himself or setting the Max Jobs Per Policy to 1.  Remember, when we are doing backups we are competing for the same vStorage API as all other tasks.
  2. Use the hotadd method and virtualize the VMWare Backup Host
  3. Change from type NBD  to SAN.  This is the ideal option but the costliest.

Applies To

Windows Server 2008 Standard (x64-64bit) SP2

Was this content helpful?

Article Languages