Vmware backup fails 'unable to quiesce file system'

Problem

VMware backups will fail with a status 156 - snapshot failed
BPFIS log will show errors quiescing the disesteem as shown below
A snapshot CAN be created from the ESX console with 'quiesce file system' enabled
A backup is successful if 'quiesce file system' is disabled from the snapshot options in the VM policy**
 

**This is not a recommended workaround as the backups are not guaranteed 'crash consistent' with quiesce disabled

Error Message

<2> onlfi_vfms_logf: INF - vSphere_freeze: freeze returned 36
and/or
<32> onlfi_freeze_fim_fs: FTL - VMware_freeze: VIXAPI freeze failed with 36
and/or
<2> onlfi_vfms_logf: INF - vmwareLogger: WaitForTaskComplete: Cannot create a quiesced snapshot because the create snapshot operation exceeded the time limit for holding off I/O in the frozen virtual machine.

Solution

This can be:
1) An issue with VSS in the guest VM:
Check for VSS errors in the event logs on the problem VM
If errors then apply the VSS rollup package from MS

2) An issue with vmware tools:
a) check that the latest version of vmware tools is installed
If not then install from ESX console (selecting interactive install will allow you to select the FULL / COMPLETE  installation NOT the TYPICAL)

b) restart the vmware tools service in the guest

c) uninstall and re-install vmware tools on the client
select FULL / COMPLETE  installation - not the TYPICAL installation

3) A high I/O guest
adjust the snapshottimeout registry setting on the VM Backup Host under
HKEY_LOCAL_MACHINE >SOFTWARE> VERITAS > NETBACKUP> CURRENTVERSION > CONFIG>BACKUP

4) If all above actions checked then refer to VMware to verify additional issues