nbdeployutil takes a long time to run, can appear to be hung

Problem

After upgrading NetBackup to 8.1.2, 8.2 or installing the overlap detection feature (KB article 100040123), nbdeployutil takes a long time to run, and can appear to be hung. We have seen both the incremental and manual execution of nbdeployutil take from 24 hours to 15 days to complete.

Error Message

No error message is encountered.

Cause

When a client appears to be in more than one policy with the same backup selection nbdeployutil needs to determine if data is being backed up more than once (overlap detection). Starting with NetBackup client 8.1 NetBackup tracks this information in the database.

However, if the client is a version prior to 8.1, then nbdeployutil needs to run bpflist/nbdgather to gather backup file lists and compare them. Because of the thousands/millions of files that need to be compared this can take significantly more time. See KB article 100040123 - Known limitations and workarounds for more details.

Parameters impacting the time required:

• multiple policies for a client having same policy-type
• number of entries (granular backup selections) getting backed-up
• frequency of full/ubak backups.

When significant gather time is observed, it is likely to be due to following policy types:

1. Standard
2. MS-Windows
3. NDMP

(Reason being number of entries getting backed up.)

Solution

For nbdeployutil to work affectively all clients will need to be updated to 8.1 or newer. The enhancement has been implemented in NetBackup client version 8.1.

• VxUpdate deployment management has been introduced to NetBackup 8.1.2 and will help streamline client upgrades.
• Native client packaging and templates for Third-party mass-deployment tools (Chef/SCCM client deployment templates) have been made available. These are important tools that helps accelerate the progress of Client upgrades.

Disable overlap detection

Upgrading clients to 8.1 or later is the recommended approach for accurate usage reporting. You can disable the overlap detection that will help generate the nbdeployutil report faster. There is no overlap detection for NetBackup client version 8.0 and earlier and will display higher capacity usage for clients with overlapping policies.

This higher usage will also be displayed in the Smart Meter portal.

To disable overlap detection for NetBackup client version 8.0 and earlier, create the following file on the master server.

Windows path: install_path\Veritas\NetBackup\var\global\63B07D3E-6D28-40B2-B124-88101B7FB39E.txt
Linux/Unix path: /usr/openv/var/global/63B07D3E-6D28-40B2-B124-88101B7FB39E.txt

Enter the following configuration parameters in the created file:

[NBDEPLOYUTIL_FEATURES]
ENABLE_BPFLIST_FEATURE=0

Note: For a scheduled run, to disable overlap processing for already gathered directories, you need to delete existing bpflist gather directories.

On Windows

• Check if you have any existing bpflist gather directories.

forfiles /s /p "install_path\Veritas\NetBackup\var\global\incremental" /m bpflist_data /c "cmd /c echo @path"

• Delete any existing bpflist gather directories.

forfiles /s /p "install_path\Veritas\NetBackup\var\global\incremental" /m bpflist_data /c "cmd /c rd /s /q @path"

Note: Ignore the following error: “ERROR: The system cannot find the file specified”. Run the command to check existing directories and confirm that the existing gather directories are deleted.

On Linux/Unix

• Check if you have any existing bpflist gather directories.

find /usr/openv/var/global/incremental/ -name bpflist_data

• Delete any existing bpflist gather directories.

find /usr/openv/var/global/incremental/ -name bpflist_data -exec rm -rf {} \;

Ignore the following error: “No such file or directory” error messages”. Run the command to check existing directories and confirm that the existing gather directories are deleted.

Please Note: If the incremental nbdeployutil takes longer than 7 days, you will need to adjust the FREQUENCY_IN_DAYS parameter or multiple incremental instances will run. See admin guide for directions.

How to determine if nbdeployutil is still running:

See if the nbdeployutil-gather-capacity is still logging, there can be long delays in the log so it is suggested to check, then check again several hours later.

Location of where to find the nbdeployutil-gather-capacity-<date>_<number>.log file:

For Incremental nbdeployutil executions:

Windows path: install_path\Veritas\NetBackup\var\global\incremental\<date>_<number>_<master_name>\nbdeployutil-gather-capacity-<date>_<number>.log
Linux/Unix path: /usr/openv/var/global/incremental/<date>_<number>_<master_name>/nbdeployutil-gather-capacity-<date>_<number>.log

For manual nbdeployutil executions:

Windows path: install_path\Veritas\NetBackup\var\global\reports\<date>_<number>_<master_name>\nbdeployutil-gather-capacity-<date>_<number>.log
Linux/Unix path: /usr/openv/var/global/reports/<date>_<number>_<master_name>/nbdeployutil-gather-capacity-<date>_<number>.log