After upgrading to NetBackup 220.127.116.11, virtual machine backups (VMware) run multiple times, eventually failing with a status 196 when the backup window closes.
Status 196. The activity monitor will show the parent job of individual backups failing in preprocessing:
06/27/2012 10:31:50 - begin Application Snapshot: Policy Execution Manager Preprocessed
Operation Status: 196
The <install_path>/openv/netbackup/db/failure_history/log.mmddyy files will show the VMware client name as the 2nd entity in the file after the date/time stamp. This entry should be the VMware backup host.
Policy Name: ESX_BACKUP
Client/HW/OS/Pri/DMI: <VM-NAME> vmx-04 rhel5_64Guest 0 0 0 0 ?
VM proxy backup host: BACKUP_SERVER1
But the failure history only shows entries for <VM-NAME> (it should be BACKUP_SERVER1):
06/27/12 00:06:47 <VM-NAME> ESX_BACKUP Differential-Inc 196 *NULL* 842 1340770007 [VM-NAME]
06/27/12 00:06:47 <VM-NAME> ESX_BACKUP Differential-Inc 196 *NULL* -1 1340770007 [VM-NAME]%5fSnapshot
06/27/12 00:20:46 <VM-NAME> ESX_BACKUP Differential-Inc 196 *NULL* 842 1340770846 [VM-NAME]
06/27/12 00:20:46 <VM-NAME> ESX_BACKUP Differential-Inc 196 *NULL* -1 1340770846 [VM-NAME]%5fSnapshot
This problem is caused by the use of the incorrect usage of the VMware client name in the failure_history log files versus the VMware backup host name.
The formal resolution for this issue (Etrack 2836511) is included in the following release:
- NetBackup 7.5 Maintenance Release 4 (18.104.22.168)
Information on NetBackup 22.214.171.124 is now available in the Related Article linked below.
An Emergency Engineering Binary (EEB) replacement for nbpem is available for NetBackup 126.96.36.199 which contains a fix for this issue. To obtain this EEB, please access the Related Article linked below.
1. Stop NetBackup services on master.
2. Remove files from /usr/open/netbackup/db/failure_history (after getting copies of the logs for advancement)
3. Remove file pempersist:
4. Restart services.
*** Note this is a temporary work around only. The problem will resurface as soon as a VMware backup failes for any reason.