V-79-57344-38277 : VMware Agent backup job of a virtual machine fails with error: You do not have access rights to this file

Problem

Error Message

V-79-57344-38277 - Unable to open a disk of the virtual machine.  VixDiskLib_Open() reported the error: You do not have access rights to this file.
Final error: 0xe0009585 - Unable to open a disk of the virtual machine.

UMI Code: V-79-57344-38277

Solution

Scenario A

Debug log:
 
From SERVERNAME-BERemoteXX.log on the Backup Exec Backup Exec Media Server:
 
VM_VCBPROXY_FS::GetVmdkDiskBitMap: status CODE (E0009585) for VM <(DC)datastore(DC)\vm\virtual_machine_name>
[6516] 7 VM_VCBPROXY_FS::GetDiskChangedInfo: GetDiskChangedInfo for VM=(DC)datastore(DC)\vm\virtual_machine_name, snapName=VM-FULL , changeId=*, DiskName=[XXX_datastore2] virtual_machine_name/virtual_machine_name.vmdk, Key=2000, offset=0
[6516] 8 SymVmTools: IsChangeTrackingEnabled: Enabled = true, Supported = false
[6516] 8 SymVmTools: GetDiskChangedInfo: entered: deviceKey = 2000, snapshotName=XXX-FULL 11-10-2010 15:10, vmPath=(DC)datastore(DC)\vm\virtual_machine_name
[6516] 8 SymVmTools: GetDiskChangedInfo: SYM_VMC_ERROR:  SOAP_ERROR
[6516] 8 SymVmTools: SOAP 1.1 fault: "":ServerFaultCode [no subcode]
"Error caused by file /vmfs/volumes/4a96ae97-514268fe-16dd-00237de19c24/virtual_machine_name/virtual_machine_name.vmdk"
Detail:
____________
 
[6516] 8 Going to perform pre backup dbcc.
[6516] 8 VM_VCBPROXY_FS::GetSelectedObjInfo() Informational status FS_AT_ROOT (E000FE0A)
[6516] 9 Windows Server 2003 (or higher) detected. No S2LP mapping required.
[6516] 9 VM_VCBPROXY_FS::GetSelectedObjInfo() Informational status FS_AT_ROOT (E000FE0A)
[6516] 9 VM_VCBPROXY_FS::GetSelectedObjInfo() Informational status FS_AT_ROOT (E000FE0A)
[6516] 9 VM_VCBPROXY_FS::OpenObj() Could not open the disk '[XXX_datastore2] virtual_machine_name/virtual_machine_name.vmdk' Error Text: 'You do not have access rights to this file' Error: '13'
[6516] 9 Cancelling the additional phase request.
[6516] 9 LP_ENV::MsgError: error 0xe0009585 processing object virtual_machine_name\virtual_machine_name.vmdk
[6516] 9 VM_VCBPROXY_FS::OpenObj() Could not open the disk '[XXX_datastore2] virtual_machine_name/virtual_machine_name_1.vmdk' Error Text: 'You do not have access rights to this file' Error: '13'
[6516] 9 Cancelling the additional phase request.
[6516] 9 LP_ENV::MsgError: error 0xe0009585 processing object virtual_machine_name\virtual_machine_name_1.vmdk
[6516] 9 VM_VCBPROXY_FS::OpenObj() Could not open the disk '[XXX_datastore2] virtual_machine_name/virtual_machine_name_2.vmdk' Error Text: 'You do not have access rights to this file' Error: '13'
[6516] 9 Cancelling the additional phase request.
[6516] 9 LP_ENV::MsgError: error 0xe0009585 processing object virtual_machine_name\virtual_machine_name_2.vmdk

Cause:
This issue may occur in the following situations:

1) If the BESA (Backup Exec Service Account) account doesn't have correct permissions on Vcenter or ESX server.

2) If the LUNs are not presented to the Media server.

Solution: 
There are multiple solutions to this error.
 
1. Verify the user account that is selected for this backup has all the correct permissions in VCenter or on the ESX host. 
     A. Please select a 'root' account if virtual machines are selected  from under an ESX host in the backup selection list.
     B. Please select a domain account with administrative privileges on the virtual machines if  virtual machines are selected  from under a VCenter server in the backup selection list.
 
3. If SAN transport mode is used, verify the drives are presented to the Backup Exec media server properly. 

Scenario B

Error from the debug log on the media server:

VDDK-Log: SAN: Cannot access a SAN/iSCSI LUN backing this virtual disk. (Hint: If you are using vcbMounter you can use the option "-m nbd" to switch to network based disk access if this is what you want.) If you were attempting file-level access, stop the vmount Service by typing "net stop vmount2" on a command prompt to force vmount to re-scan for SAN LUNs and re-try the command.
VDDK-Log: DISKLIB-LINK  : "san://snapshot[DATASTORE] SERVERNAME/SERVERNAME.vmdk@ESXHOST?DATA\USERNAME/XXX" : failed to open (You do not have access rights to this file).
VDDK-Log: DISKLIB-CHAIN : "san://snapshot[DATASTORE] SERVERNAME/SERVERNAME.vmdk@ESXHOST?DATA\USERNAME/XXX" : failed to open (You do not have access rights to this file).
VDDK-Log: DISKLIB-LIB   : Failed to open 'san://snapshot[DATASTORE] SSERVERNAME/SERVERNAME.vmdk@ESXHOST?DATA\USERNAME/XXX' with flags 0x1e (You do not have access rights to this file).
VM_VCBPROXY_FS::OpenObj() Could not open the disk '[DATASTORE] SERVERNAME/SERVERNAME.vmdk' Error Text: 'You do not have access rights to this file' Error: '13'

Cause:
When backing up a virtual machine using only SAN transport mode the backup can fail with the above error message.  This error can be caused by any of the following.

1. The media server performing the backup does not have access to the SAN.
2. The media server performing the backup can not resolve the name of the ESX server.
3. The user account specified in the backup job does not have all necessary rights needed to perform the backup.

Solution:
1. Verify the media server can see the SAN disks and they are not marked as OFFLINE.
2. Ping the ESX Host by name from the media server.  Verify the media server can resolve the name and IP of the ESX host correctly.
3. Create a new user account and give it administrator rights in vCenter or the ESX host and attempt the backup again using the new user account.

 

Scenario C

Debug error:

VDDK-Warn: [NFC ERROR] NfcFssrvr_DiskOpen: received NFC error 11 from server: NfcFssrvrOpen: permission check failed for file [DataStore]Virtual_Machine_Name/Virtual_Machine_Name.vmdk, access = 1
VDDK-Log: DISKLIB-DSCPTR: : "vpxa-nfc://[DataStore] Virtual_Machine_Name/Virtual_Machine_Name.vmdk be 13 67 00 94 6e 91-e0 d0 3f ce 9c 11 62 b6" : Failed to open NBD extent.
VDDK-Log: DISKLIB-LINK  : "vpxa-nfc://[DataStore] Virtual_Machine_Name/Virtual_Machine_Name.vmdk be 13 67 00 94 6e 91-e0 d0 3f ce 9c 11 62 b6" : failed to open (NBD_ERR_GENERIC).
VDDK-Log: DISKLIB-CHAIN : "vpxa-nfc://[DataStore] Virtual_Machine_Name/Virtual_Machine_Name.vmdk be 13 67 00 94 6e 91-e0 d0 3f ce 9c 11 62 b6" : failed to open (NBD_ERR_GENERIC).
VDDK-Log: DISKLIB-LIB   : Failed to open 'vpxa-nfc://[DataStore] Virtual_Machine_Name/Virtual_Machine_Name.vmdk be 13 67 00 94 6e 91-e0 d0 3f ce 9c 11 62 b6' with flags 0x1e (NBD_ERR_GENERIC).

Cause:
This issue might occur if there are multiple VSS providers are installed on the Virtual machine

Eg:VMware Snapshot Provider and Backup Exec VSS Provider are installed on a virtual machine.

Soluton:
Uninstall the VMware Snapshot Provider (instructions in the link below)
https://www.veritas.com/docs/100001832

 

Scenario D

There may be a problem with DNS.

Cause:
3-node VMware vSphere cluster (nodes named esx1, esx2, esx3)

Error comes up only when migrating VMs to the esx3 host.

Solution:
Add the appropriate DNS record for the problematic host.

Example : Add adding the appropriate DNS record esx3.mynetwork.lan for the problematic host esx3

 

Scenario E

Debug error:

[7104] 2013-11-19T11:46:42.395 [fsys\shared]        - VDDK-Log: VixDiskLib: Trying to open disk [Volume2] ExchangeWeb/ExchangeWeb.vmdk using mode nbd.
[7104] 2013-11-19T11:46:42.395 [fsys\shared]        - VDDK-Log: NBD_ClientOpen: attempting to create connection to vpxa-nfc://[Volume2] VMName/VMName.vmdk@vmhost.domain.local:902
[7104] 2013-11-19T11:46:42.395 [fsys\shared]        - VDDK-Log: Started up WSA
[7104] 2013-11-19T11:46:42.452 [fsys\shared]        - VDDK-Log: CnxAuthdConnect: Returning false because CnxAuthdConnectTCP failed
[7104] 2013-11-19T11:46:42.453 [fsys\shared]        - VDDK-Log: CnxConnectAuthd: Returning false because CnxAuthdConnect failed
[7104] 2013-11-19T11:46:42.453 [fsys\shared]        - VDDK-Log: Cnx_Connect: Returning false because CnxConnectAuthd failed
[7104] 2013-11-19T11:46:42.453 [fsys\shared]        - VDDK-Log: Cnx_Connect: Error message: Host address lookup for server vmhost01.gilman.edu failed: The requested name is valid, but no data of the requested type was found
[7104] 2013-11-19T11:46:42.453 [fsys\shared]        - VDDK-Warn: [NFC ERROR] NfcNewAuthdConnectionEx: Failed to connect to peer. Error: Host address lookup for server vmhost.domain.local failed: The requested name is valid, but no data of the requested type was found

Cause:
This issue might occur if Backup Exec cannot access the ESX hosts by the method provided by the vCenter server.

Solution:
Correct the DNS issues causing the failed name resolution