During VMware backup of Redhat Linux guest with quiesce enabled, the VM will hang and become unresponsive

  • Modified Date:
  • Article ID:000021419

Problem

When a VMware backup with quiesce enabled takes place, within the VM a filesystem sync and freeze operation is called prior to the VMware Snapshot to ensure the snapshot is consistent. It has been discovered that during the filesystem freeze operation, certain RedHat versions will lockup or hang during the freeze of the filesystem. The VM will also not allow any further logins. The only way to recover is to reset the VM. 

Error Message

NetBackup reports a successful Status 0 backup in the Activity Monitor yet the VM guest becomes unresponsive during the quiesce operation.
Subsequent backup and snapshot attempts also report as successful in NetBackup yet the VM remains in a hung or unresponsive state.

Cause

Certain kernel versions of RedHat Enterprise Linux have a known problem with the vmsync driver causing a deadlock if vmsync is used during a VMware Snapshot with quiesce enabled. The source of the deadlock occurs because of a problem with the FIFREEZE/FITHAW ioctl() feature within the Linux guest that is utilized to quiesce the filesystem.

For more information see:
https://kb.vmware.com/kb/2038606

Solution

During the install of VMware Tools for Linux, the 'vmsync' driver is still listed as "Experimental". For Linux VMs VERITAS recommends using VMware Tools File System Sync Driver.

If you would like to continue using SYMCquiesce, then vmsync must be removed as they cannot work together performing the same operation.

SYMCquiesce is available as an ISO download in HOWTO92298.


Steps to Disable VMware vmsync driver

To prevent the vmsync driver from being called during the quiesce phase of a VMware snapshot, edit the VMware Tools configuration file as follows:

1) Open a console session to the Redhat Linux virtual machine.
2) Navigate to the /etc/vmware-tools directory
3) Using a text editor, modify the tools.conf file with the following entry

[vmbackup]
enableSyncDriver = false

Note: If the tools.conf file does not exist, create a new empty file and add the above parameters.



VMware Knowledge Article: 

Snapshot quiescing fails on Linux guests after upgrading to ESXi and VMware Tools 5.1 and 5.5 (2038606)

http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=2038606


RedHat Knowledge Articles:

System hangs when taking quiesced snapshot in VMware Environment.
https://access.redhat.com/solutions/1257523

System Hangs during VMware Snapshot operation
https://access.redhat.com/solutions/484303

Red Hat Enterprise Linux 6 guest running on VMware® Virtual Platform hangs while creating snapshots
https://access.redhat.com/solutions/372013

Customers looking for more information about the issue should contact RedHat Support and reference BUG ID: 768304.


 

Applies To

RedHat Enterprise Linux 5 - All Versions
RedHat Enterprise Linux 6 - 6.4 (kernel 2.6.32.358) and earlier
VMware tools with vmsync driver installed
SYMCquiesce may or may not be installed
Other Linux distributions may also be affected by this issue.

Terms of use for this information are found in Legal Notices.

Search

Survey

Did this article answer your question or resolve your issue?

No
Yes

Did this article save you the trouble of contacting technical support?

No
Yes

How can we make this article more helpful?

Email Address (Optional)