nbdeployutil - Overlap detection feature for NetBackup versions 7.7.2, 7.7.3, 8.0, 8.1, and 8.1.1

nbdeployutil - Overlap detection feature for NetBackup versions 7.7.2, 7.7.3, 8.0, 8.1, and 8.1.1

Article: 100040123
Last Published: 2019-01-23
Ratings: 5 1
Product(s): NetBackup

Description

The new version of nbdeployutil introduces the Overlap Detection feature. This feature is applicable only for Capacity Licensing. With this feature, nbdeployutil detects if the same data is backed up by multiple policies of the same policy type and deducts the charged size for overlapping backup selections. nbdeployutil also supports the obfuscation feature that provides a command line option to anonymize the master server name and the client name in the report.

This article has the following sections:

Supported NetBackup versions and agents

Feature Supported NetBackup versions Supported NetBackup agents
Overlap Detection
  • 7.7.2
  • 7.7.3
  • 8.0
  • 8.1
  • 8.1.1
  • File System (Standard and Microsoft Windows)
  • Microsoft SQL Server
  • NDMP
  • VMware
  • Microsoft Exchange Server

Version-specific support

NetBackup master server NetBackup agents, policy types, or workloads
Require EEBs to report accurate license usage Report accurate license usage
7.7.2 NetBackup client version: 7.7.2
FS, SQL, Exchange Server, Vmware, NDMP
-
7.7.3 NetBackup client version: 7.7.2 & 7.7.3
FS, SQL, Exchange Server, Vmware, NDMP
-
8 NetBackup client version: 7.7.2 , 7.7.3, & 8.0
FS, SQL, Exchange Server, Vmware, NDMP
-
8.1 NetBackup client version: 7.7.2 , 7.7.3, & 8.0
FS, SQL, Exchange Server, Vmware, NDMP
NetBackup client version: 8.1
FS, SQL, Exchange Server, Oracle, Vmware, NDMP, BigData
8.1.1 NetBackup client version: 7.7.2 , 7.7.3, & 8.0
FS, SQL, Exchange Server, Vmware, NDMP
NetBackup client version: 8.1 & 8.1.1
FS, SQL, Exchange Server, Oracle, Vmware, NDMP, BigData

The Overlap Detection feature includes the Incremental Reporting feature of nbdeployutil. To know more about the Incremental Reporting feature, refer to the following article:
http://www.veritas.com/docs/000109508

Installing the Emergency Engineering Binary

Refer to the Download section to download the nbdeployutil zip file based on the version of NetBackup that you have installed.
Use the following steps to get the latest version of nbdeployutil with the Overlap Detection feature:
  1. From the command prompt, navigate to the location of the downloaded nbdeployutil zip file.
  2. Run the following command:
    <eebinstaller> -create
    Where <eebinstaller> is the name of the extracted file.

How the Overlap Detection feature works

When you run nbdeployutil, either manually or using the Incremental Reporting feature, overlapping backup selection data is detected and automatically adjusted in the Charged Size column for the overlapping policies in the report. This feature ensures that customers are charged optimally and are not charged for duplicate backed up data for the File System (Standard and Microsoft Windows) and Microsoft SQL Server policy types.

How to use nbdeployutil

You can use the existing command line options while running nbdeployutil.
For more information, refer to the following article:
https://www.veritas.com/content/support/en_US/doc/123533878-127136857-0/v123549289-127136857

Examples

The following examples give an overview of the overlap cases and display how the overlap is reported.

Example 1: Policy Type - File System Standard and Microsoft Windows

Policy 1 Policy 2
Backup Selection:
  • C:\Data1 [Size 4GB]
  • C:\Data2 [Size 2GB]
Backup Selection:
  • C:\Data2 [2GB]
  • C:\Data3 [1GB]
In this example, the C:\Data2 backup selection is common in both Policy 1 and Policy 2. nbdeployutil detects the overlap and displays in the report. An extract of the report specific to this example is as follows:
Policy Name Accuracy Accuracy Comment Overlap
Size (GB)
Total (Readable)
Size (GB)
Charged
Size (GB)
Policy 1 Ok   0 6 6
Policy 2 Ok Overlap detected for the policy and deducted from the Charged Size 2 3 1
For Policy 1 the customer is charged for C:\Data1 and C:\Data2. For Policy 2, the customer is charged only for C:\Data3.

Example 2: Policy Type - Microsoft SQL Server

Policy 1: DB_File_Overlap1 Policy 2: DB_File_Overlap2
Backup Selection: 
  • Instance=INST_R  DB file=DB_01 [Size 1.4GB]
  • Instance=INST_R  DB file=DB_02 [Size 1.4GB]
Backup Selection:
  • Instance=INST_R  DB file=DB_02 [Size 1.4GB]
  • Instance=INST_R  DB file=DB_03 [Size 1.4GB]
In this example, the DB file DB_02 is common in both policies DB_File_Overlap1 and DB_File_Overlap2. nbdeployutil detects the overlap and displays in the report. An extract of the report specific to this example is as follows:
Policy Name Accuracy Accuracy Comment Overlap
Size (GB)
Total (Readable)
Size (GB)
Charged
Size (GB)
DB_File_Overlap1 Database Estimation, Multistream Estimation, OK DB size estimated via backup summation, Multi-stream backup estimation 0 2.8 2.8
DB_File_Overlap2 Database Estimation, Multistream Estimation, OK DB size estimated via backup summation, Multi-stream backup estimation, Overlap detected for the policy and deducted from the Charged Size 1.4 2.8 1.4
For policy DB_File_Overlap2 the customer is charged for only DB file DB_03.

Example 3: Policy Type - VMware

Policy 1 Client: WIN2012-HN.RSV.VEN.veritas.com
VM Primary Identifier: VM BIOS UUID
Backup Selection: Exclude boot disk [Size 4GB]
Policy 2 Client: WIN2012-HN.RSV.VEN.veritas.com
VM Primary Identifier: VM INSTANCE UUID
Backup Selection: Include all disks [Size 12GB]
Policy 3 Client: WIN2012-HN.RSV.VEN.veritas.com
VM Primary Identifier: VM display name
Backup Selection: Exclude boot disk [Size 4GB]
Policy 4 Client: WIN2012-HN.RSV.VEN.veritas.com
VM Primary Identifier: VM DNS name
Backup Selection: Exclude data disk [Size 8GB]
In this example, all the four policies are for the same client, but are made by selecting different primary identifiers. Here, Policy 2 includes both the disks so it is consuming all the other policies. Therefore, only Policy 2 is charged.
An extract of the report specific to this example is as follows:
Policy Name Accuracy Accuracy Comment Overlap Size (GB) Total (Readable Size (GB) Charged Size (GB)
Policy 1 OK Modified size using file/policy data (removed overlapping files) 0 4 0
Policy 2 OK   0 12 12
Policy 3 OK Modified size using file/policy data (removed overlapping files) 0 4 0
Policy 4 OK Modified size using file/policy data (removed overlapping files) 0 8 0
Only Policy 2 is charged as it consumes the other three policies.

Example 4: Policy Type - NDMP

Policy 1 Policy 2
Backup Selection:
  • /NDMP_Filer/Volume1 [Size 4GB]
  • /NDMP_Filer/Volume2 [Size 2GB]
Backup Selection:
  • /NDMP_Filer/Volume2 [2GB]
  • /NDMP_Filer/Volume3 [1GB]
In this example, the /NDMP_Filer/Volume2 backup selection is common in both Policy 1 and Policy 2. nbdeployutil detects the overlap and displays in the report.
An extract of the report specific to this example is as follows:
Policy Name Accuracy Accuracy Comment Overlap Size (GB) Total (Readable Size (GB) Charged Size (GB)
Policy 1 OK   0 6 6
Policy 2 OK Overlap detected for the policy and deducted from the Charged Size 2 3 1
For Policy 1, the charges are applicable for /NDMP_Filer/Volume1 and /NDMP_Filer/Volume2. For Policy 2, the charges are applicable only for /NDMP_Filer/Volume3.

Example 5: Microsoft Exchange Server

Policy 1 Policy 2
Backup Selection:
  • Microsoft Information Store:\DB1
  • Microsoft Information Store:\DB2
Backup Selection:
  • Microsoft Information Store:\DB1
  • Microsoft Information Store:\TestDB
In this example, the backup selection Microsoft Information Store:\DB1 is common in both Policy 1 and Policy 2. nbdeployutil detects the overlap and displays in the report.
An extract of the report specific to this example is as follows:
Policy Name Accuracy Accuracy Comment Overlap Size (KB) Duplicate Selection Total (KB) Charged Size (KB)
Policy 1 Database Estimation, OK DB size estimated via backup summation, Overlap detected for the policy and deducted from the Charged Size. 8260 /Microsoft Information Store/DB1/Database 39309 31049
Policy 2 Database Estimation, OK DB size estimated via backup summation 0   41362 41362
For Policy 1, the charges are applicable only for Microsoft Information Store:\DB2. For Policy 2, the charges are applicable for both Microsoft Information Store:\DB1 and Microsoft Information Store:\TestDB.

Example 6: Microsoft Exchange Server (Consumption scenario)

Policy 1 Backup Selection:
  • Microsoft Information Store:\DB1
  • Microsoft Information Store:\DB2
Policy 2 Backup Selection:
  • Microsoft Information Store:\DB1
  • Microsoft Information Store:\TestDB
Policy 3 Backup Selection:
  • Microsoft Information Store:\
In this example, the backup selection Microsoft Information Store:\ directive includes all the databases present in the Microsoft Exchange client. nbdeployutil detects the consumption and displays in the report.
An extract of the report specific to this example is as follows:
Policy Name Accuracy Accuracy Comment Total (KB) Charged Size (KB)
Policy 1 Database Estimation DB size estimated via backup summation, Modified size using file/policy data (removed overlapping files) 39309 0
Policy 2 Database Estimation DB size estimated via backup summation, Modified size using file/policy data (removed overlapping files) 41362 0
Policy 3 Database Estimation DB size estimated via backup summation 109597 109597
For Policy1 and Policy2, the customer is not charged. For Policy3, the charges are applicable for Microsoft Information Store:\ that includes the DB1, DB2, TestDB and other available databases in that client.

Additional options for nbdeployutil that are available with the Overlap Detection feature

CLI option Option for Incremental Reporting feature Description
--overlap-details In the nbdeployutilconfig.txt file, add SHOW_OVERLAP_DETAILS=1

For more information, see
http://www.veritas.com/docs/000109508
Use this option while generating the capacity licensing report to display duplicate backup selections in the Duplicate Selections column of the report.

Note:
  • Use this option only for ASCII or English-only characters in the backup selection data.
  • The following disclaimer is displayed when you run this option:

    The overlap-details flag creates the Duplicate Selection column with data, which could be potentially sensitive.
--disable-overlap        In the nbdeployutilconfig.txt file, add  DISABLE_OVERLAP_DETECTION=1 

For more information, see
http://www.veritas.com/docs/000109508
Use this option along with the existing gather and report options to disable the Overlap Detection feature.
--anonymize   Use this option to anonymize the master server name and client names in the report. By default, the master server names are replaced with MASTER_{X} and client names are replaced with HOST_{X}, where X is a numeric value.
--anonymize -anon-master=SOMENAME   Use this option to anonymize the master server name and client names in the report. The master server name is replaced with SOMENAME and the client name is replaced with HOST_{X}, where X is a numeric value.

Information that is displayed for the anonymize option in the report

For the following command:
nbdeployutil.exe --gather --capacity --report

The report is displayed as:
Master Server Client Name Policy Name Policy Type
win-b2qj6keaalf 5035be4e-1ec8-f99c-34fd-e2c16110e96e Policy 1 VMware

For the following command:
nbdeployutil.exe --gather --capacity --report --anonymize

The report is displayed as:
Master Server Client Name Policy Name Policy Type
Master_0 Host_1 Policy 1 VMware

For the following command:
nbdeployutil.exe --gather --capacity --report --anonymize -anon-master=SOMENAME

The report is displayed as:
Master Server Client Name Policy Name Policy Type
SOMENAME_0 Host_1 Policy 1 VMware

Information that is displayed for the duplicate Microsoft SQL backup selections in the report

In the generated report for File System and Microsoft SQL policies, OK is displayed in the Accuracy column when an overlap is detected and the charged size is deducted.
The duplicate backup selection for Microsoft SQL Server has the following format:
<InstanceName>/<DbName>/<ObjectType>:<ObjectName>
For example, /Instance1/DB2/filegroup:testgroup.

In case of Availability Group dbName is prefixed by agname in the following format:
<InstanceName>/<AgName>:<DbName>/<ObjectType>:<ObjectName>
For example, /<defaultInstance>/AGTEST1:DB1/.

Known limitations and workarounds

  • The feature cannot deduct overlap size in the following scenarios:
    • Backup policy contains the following directives:
      • "System-State"
      • "Shadow-Copy components"
    • Compression is enabled for backups.

  • The overlap size is not deducted from the charge size and Accuracy is mentioned as Overlap in the report instead of OK.
  • The Overlap Detection feature gathers more information for optimally calculating the charged size and might require more disk space than the earlier version of nbdeployutil. Use the --output <LOCATION> option along with the --gather option to redirect the gather output to a different location.
  • If you have enabled catalog compression, nbdeployutil decompresses the catalog and the disk space usage increases. To run this utility effectively, ensure that there is sufficient disk space available. To frequently compress the catalog, set the Compress catalog interval option to a minimal value. For example, you can set it as one day. This setting ensures that the uncompressed files are not stored for a longer duration after you run nbdeployutil. For more information, refer to https://www.veritas.com/content/support/en_US/doc/123711489-126541862-0/v41242926-126541862.
  • The Overlap Detection feature might take significantly more time to effectively gather, calculate, deduct, and then display the charged size for the overlapping backup policies. During this process, there is no other effect on your production environment and on the tasks running on the master server. However, you can also run the report option of nbdeployutil on a different computer.
  • Overlap Detection feature is not supported from OpsCenter.

Updates to the nbdeployutil EEBs

  1. Detect and report data overlap for File System, SQL, VMware, NDMP and MS Exchange workloads.
  2. Incrementally gather and report capacity usage information.
  3. Provide the ability to anonymize host names in the output report
  4. Generate a UsageData_Sanitized.out file that provides the capacity (front-end data) usage information to the NetBackup telemetry.
    Note: The following fields are obfuscated in the file: policy name, client name, master server name

 

Was this content helpful?