VMware Backup fails with status 6 and 23.

Problem

VM client failing with a status 6/23 in the job details while attempting to use SAN or HOTADD transport.
When the backup job runs it would fail almost immediately with errors shown below.

• Backup via NBD transport will work
• May still fail after vMotion of VM to a different Datastore and ESX host

Error Message

from Detailed Status:

10/21/2013 11:18:28 - Info bptm (pid=14932) using 1048576 data buffer size
10/21/2013 11:18:28 - Info bptm (pid=14932) setting receive network buffer to 1048576 bytes
10/21/2013 11:18:28 - Info bptm (pid=14932) using 512 data buffers
10/21/2013 11:18:29 - Info bptm (pid=14932) start backup
10/21/2013 11:18:32 - begin writing
10/21/2013 11:18:34 - Error bpbrm (pid=14922) from client winclient.local: ERR - Error opening the snapshot disks using given transport mode: Status 23
10/21/2013 11:18:37 - Error bptm (pid=14932) media manager terminated by parent process
10/21/2013 11:18:42 - Info backup2 (pid=14932) StorageServer=PureDisk:backup2; Report=PDDO Stats for (backup2): scanned: 3 KB, CR sent: 0 KB, CR sent over FC: 0 KB, dedup: 100.0%
10/21/2013 11:18:42 - Info bpbkar (pid=0) done. status: 6: the backup failed to back up the requested files
10/21/2013 11:18:42 - end writing; write time: 0:00:10
the backup failed to back up the requested files (6)

VXMS logging may show errors similar to:

• Could not unmount /tmp/vmware-root/423223da-8254-debb-7aed-dd8b774d8ab7-vm-1652/san: Virtual Machine with MoRef "vm-1652" has no snapshot..
• Attempting to clean mount dir /tmp/vmware-root/42323648-21ab-9258-8f8d-50ddf41c8d14-vm-2521.
• Trying to unmount /tmp/vmware-root/42323648-21ab-9258-8f8d-50ddf41c8d14-vm-2521/san.

Cause

Problem was caused by orphaned snapshot mount point and lock files on the appliance or the backup/access host (media server) if NOT an appliance.

Solution

Before making any changes please check to confirm that there are no backups, restores, or stale processes running on the media server.

With no backups running you may delete the files in the /tmp/vmware-root directory.
The same files on a Windows Backup host reside in the C:\Windows\Temp\vmware-SYSTEM<ID_Name>\ directory.

These files should / can be deleted when this backup is not active. These files will get recreated when running new VMware backups.