A Windows NetBackup Client upgrade fails and rolls back

Problem

The upgrade of an existing NetBackup 7.x.x - 8.x.x Client proceeds to the point where it displays the following messages:

Current Step: Removing backup files
File: Waiting for command: icacls



The install hangs on this screen for over a minute, then the install fails and a Rollback occurs.

 

Error Message

The NetBackup Install Log produced by the install attempt captures the following point of failure:

NOTE: Install Logs are located here: C:\ProgramData\Symantec\NetBackup\InstallLogs

11-10-2016,16:48:54 :  Running command: icacls "C:\Program Files\Veritas\NetBackup\Logs\user_ops" /remove:g *S-1-1-0 /t.
11-10-2016, 16:48:54 :  File: Waiting for command: icacls
11-10-2016, 16:50:05 :  CustomAction Commit_CorrectUserOpsPermissions returned actual error code 1603 (note this may not be 100% accurate if translation happened inside sandbox)
11-10-2016,16:50:05 :  Action 16:50:05: Rollback. Rolling back action:


At the same point in time, the Application Event Log documents a fault of Microsoft's MsiExec.exe process (Windows installer):

Thu Nov 10 2016 16:50:05    Windows Error Reporting    I1001    Fault bucket , type 0
Event Name: APPCRASH
Response: Not available
Cab Id: 0
Problem signature:
P1: MsiExec.exe
P2: 5.0.9600.17905
P3: 557f547f
P4: MSIE9DD.tmp
P5: 7.703.16.519
P6: 573e3c8c
P7: c00000fd
P8: 00000000003201b7

Cause

This issue occurs because the icacls command is taking too long to produce a return code.  The MsiExec.exe process expects a return code in under 1 minute.  When a return code is not received in a timely fashion, the process faults thus causing the installation rollback to occur.

The icacls command is attempting to adjust permissions on the files and folders residing inside: "C:\Program Files\Veritas\NetBackup\Logs\user_ops".

The reason the icacls command is taking so long to complete becomes clear upon close examination of the number of files residing inside of the "C:\Program Files\Veritas\NetBackup\Logs\user_ops" directory structure.

Within the following two sub-directories reside over 3,500 files:
C:\Program Files\Veritas\NetBackup\logs\user_ops\mssql\logs
C:\Program Files\Veritas\NetBackup\logs\user_ops\mssql\jobs


If you were to run the same icacls command outside of the install sequence, you would observe the command results scroll up the screen, and it would likely complete successfully after a few minutes time.

Solution

The solution is to remove all files which have accumulated within the following two directories:

C:\Program Files\Veritas\NetBackup\logs\ user_ops\mssql\logs
C:\Program Files\Veritas\NetBackup\logs\ user_ops\mssql\jobs

These files are produced during backups and restores of MS SQL databases and transaction logs.  The files inside the directories contribute to the Job Details and View Status details observed during a backup job.
The files are not needed or used once the NetBackup for SQL operations complete.

If no backup or restore jobs are active for the client, all of the files in these folders can be deleted. 
Note: Keep the directory structure intact.

Once the ... \user_ops sub-directory structure contains a smaller amount of files, the icacls command can complete faster and the next attempt to upgrade the client should succeed.