Virtual machine restore to vSphere 8.x datastore fails with 2820 when CPU HotPlug is enabled on the VM.
Problem
Restoring a virtual machine that has CPU Hot Plug (Hot Add) enabled in the virtual machine settings fails when directed to a vSphere 8.x datastore.
Restores to vSphere 7.x datastores are successful.
Error Message
Job Details will show something similar to the following:
Jun 11, 2023 12:03:46 PM - begin reading
Jun 11, 2023 12:03:47 PM - end reading; read time: 0:00:01
Jun 11, 2023 12:03:47 PM - Info bpdm (pid=60649) completed reading backup image
Jun 11, 2023 12:03:47 PM - Info bpdm (pid=60649) EXITING with status 0
Jun 11, 2023 12:03:47 PM - Info vrts-media1 (pid=60649) StorageServer=PureDisk:vrts-media1-; Report=PDDO Stats for (vrts-media1): read: 5579 KB, CR
received: 59390 KB, CR received over FC: 0 KB, dedup: 0.0%
Jun 11, 2023 12:03:52 PM - Info bpVMutil (pid=60666) INF - vmwareLogger: WaitForTaskComplete: Invalid virtual machine configuration. <292>
Jun 11, 2023 12:03:52 PM - Info bpVMutil (pid=60666) INF - vmwareLogger: WaitForTaskComplete: SYM_VMC_ERROR: TASK_REACHED_ERROR_STATE
Jun 11, 2023 12:03:52 PM - Info bpVMutil (pid=60666) INF - vmwareLogger: CreateVm: SYM_VMC_ERROR: TASK_REACHED_ERROR_STATE
Jun 11, 2023 12:03:52 PM - Info bpVMutil (pid=60666) INF - vmwareLogger: CreateVirtualMachineExAPI: SYM_VMC_ERROR: TASK_REACHED_ERROR_STATE
Jun 11, 2023 12:03:52 PM - end Restore; elapsed time 0:00:10
NetBackup VMware policy restore error (2820)
The BPVMUTIL log shows this error:
12:03:51.002 [60666.60666] <2> virtlogcallback: vmwareLogger: CreateVmExAPI: Completed Request dump.
12:03:51.010 [60666.60666] <2> virtlogcallback: vmwareLogger: CreateVmExAPI: OK
12:03:51.010 [60666.60666] <2> virtlogcallback: vmwareLogger: Thread <-1260472448> releasing lock in CreateVmExAPI, level 1
12:03:51.013 [60666.60666] <2> virtlogcallback: vmwareLogger: WaitForTaskComplete: reached Running...
12:03:52.009 [60666.60666] <2> virtlogcallback: vmwareLogger: WaitForTaskComplete: reached Error!
12:03:52.009 [60666.60666] <2> virtlogcallback: vmwareLogger: WaitForTaskComplete: Invalid virtual machine configuration. <292>
<snip>
3:52.021 [60666.60666] <2> virtlogcallback: vmwareLogger: WaitForTaskComplete: Starting SoapFault dump.
12:03:52.021 [60666.60666] <2> virtlogcallback: vmwareLogger: <?xml version="1.0" encoding="UTF-8"?> <SoapFault
xsi:type="ns2:InvalidVmConfig"><ns2:faultMessage><ns2:key>msg.hostd.configSpec.vnuma.onlyCpuHotadd</ns2:key><ns2:message>Virtual NUMA cannot be configured when
CPU hotadd is enabled.</ns2:message></ns2:faultMessage><ns2:property>configSpec.virtualNum
12:03:52.021 [60666.60666] <2> virtlogcallback: vmwareLogger: a</ns2:property></SoapFault>
12:03:52.021 [60666.60666] <2> virtlogcallback: vmwareLogger: WaitForTaskComplete: Completed SoapFault dump.
Cause
vNUMA is disabled if vCPU hotplug is enabled:
https://kb.vmware.com/s/article/2040375
When the VM is powered on, the vCenter will set some of the vNUMA configuration. NetBackup cannot make requests to do that. These requests are not required with vSphere 7.x.
Solution
The workaround to this issue is to disable the CPU hotplug on the VMs being backed up -OR- to restore the VMs to a vSphere 7.x datastore.
Because the problem is with the data at backup time, VMs that were backed up prior to disabling the CPU hotplug can only be restored to vSphere 7.x datastores.
A supported hotfix has been made available for this issue. Please contact Veritas Technical Support to obtain this fix. This hotfix has not yet gone through any extensive Q&A testing. Consequently, if you are not adversely affected by this problem and have a satisfactory temporary workaround in place, we recommend that you wait for the public release of this hotfix.
Veritas Technologies LLC currently plans to address this issue by way of a patch or hotfix to the current version of the software. Please note that Veritas Technologies LLC reserves the right to remove any fix from the targeted release if it does not pass quality assurance tests. Veritas’ plans are subject to change and any action taken by you based on the above information or your reliance upon the above information is made at your own risk.
Please contact your Veritas Sales representative or the Veritas Sales group for upgrade information including upgrade eligibility to the release containing the resolution for this issue.