Slow VMware restore performance for thin-provisioned windows VMs using NBD transport

Problem

The restore performs well to a certain point, but then slows down. The restore completes successfully eventually, but takes a long time in comparison to other identical windows VMs (same number of drives, number of virtual disks, installed apps and disk consumption). Same restore using SAN transport completes with acceptable performance.

Error Message

No error messages can be discovered. While the job appears to be in hung state, VxMS, BPTM, NCFNBRESTORE do show the progress in writing the data.

Cause

Highly fragmentation on Windows guest volumes.

Solution

Various troubleshooting steps can be taken to isolate this issue further.

- trying other transport modes and see the outcome.
- checking to see if VM instant recovery can be performed.
- are we getting the same outcome for other restore types (Ms-Windows, SQL DB, ...)?
- do we get the same outcome in restoring other VMs?
- do we get the same outcome in restoring Linux VMs?
- do we get any performance gains when we restore to a different datastore/esxi hosts?
- trying to duplicate the images to an advdisk and performing the restore to see if there are any performance gains?

The outcome of all the above diagnostic actions can led to the conclusion that the issue is with restoring one specific VM using NBD with no configuration disparity with other similar VMs.

Perform a defrag on the Windows guest volumes, then run a full backup to solve the issue.