UNIX/Linux NetBackup Catalog Backup fails to trigger the notification e-mail when using ‘ mail_dr_info.sh ‘

Problem

It has been observed since NetBackup 8.1 the NetBackup catalog backup now includes two files (the catalog recovery file and the disaster recovery package (*.drpkg) file), as part of the disaster recovery e-mail when a catalog backup completes.  This now causes a problem when using the script variables (in particular $4) to try an attach these files to an e-mail or redirect the contents of the files to another location.    

Within the NetBackup catalog backup policy there is a ‘Disaster Recovery’ tab which contains the option ‘Send in an E-mail attachment ( recommended )’.  If this option is checked when the catalog backup completes (successful or failed), on a UNIX/Linux master the file ‘ /usr/openv/netbackup/bin/mail_dr_info.sh ‘ is used to try and deliver the catalog backup results.

Note: By default (except on a Veritas NetBackup appliance), the file ‘ /usr/openv/netbackup/bin/mail_dr_info.sh ‘ does not exist and this must be manually created.  For further details see the “Veritas NetBackup Administrator’s Guide Volume II” (section: NetBackup notify scripts), as this provides details on the script variables which can be used.

/usr/openv/netbackup/bin/mail_dr_info.sh
NetBackup passes the following parameters to the script:

$1 Specifies the recipient's address. For multiple addresses, enter email1,email2
$2 Specifies the subject line.
$3 Specifies the message file name.
$4 Specifies the attached file name.

Error Message

Extract of the NetBackup /usr/openv/netbackup/logs/bpdbm/<logname> log showing the point of failure:
05:57:36.246 [8341] <2> send_DR_report: Entering
05:57:36.246 [8341] <2> send_DR_report: Sending DR package as attachment </NB_DR_FILE/NBCAT_MasterServerName_1617875853_FULL.drpkg> <0>
05:57:36.246 [8341] <2> send_DR_report: Start OVsystem("/usr/openv/netbackup/bin/mail_dr_info.sh" "MailUserName@MasterServerName" "NetBackup Catalog Backup successful on host 
MasterServerName status 0" "/usr/openv/netbackup/db/mail/MasterServerName_1617875853" 
"/NB_DR_FILE/NBCAT_MasterServerName_1617875853_FULL,/NB_DR_FILE/NBCAT_MasterServerName_1617875853_FULL.drpkg")
05:57:36.257 [8341] <16> send_DR_report: failed to execute "/usr/openv/netbackup/bin/mail_dr_info.sh" "MailUserName@MasterServerName" "NetBackup Catalog Backup successful on 
host MasterServerName status 0" "/usr/openv/netbackup/db/mail/MasterServerName_1617875853" 
"/NB_DR_FILE/NBCAT_MasterServerName_1617875853_FULL,/NB_DR_FILE/NBCAT_MasterServerName_1617875853_FULL.drpkg" (1)
05:57:36.258 [8341] <4> db_error_add_to_file: Could not send DR report
05:57:36.258 [8341] <16> generate_DR_file_and_send_mail: Could not send DR report
05:57:36.258 [8341] <16> exec_catalog_DR_protection: Failed to send DR image to MailUserName@MasterServerName (0)

Cause

The $4 variable contains both disaster recovery files including their paths, separated by a comma ( , ).  If using a custom email script, is it important to handle the $4 variable correctly so that it parses two files comma separated. Failing to handle both file attachments will result in an incomplete DR Email and fail any Catalog restores using the email's contents.

Example of the contents of $4:
/NB_DR_FILE/NBCAT_MasterServerName_1617877307_FULL,/NB_DR_FILE/NBCAT_MasterServerName_1617877307_FULL.drpkg
 

Solution

Veritas Support supply default example scripts which perform no actions. These scripts are templates and made available as examples (located: /usr/openv/netbackup/bin/goodies/).  The ‘/usr/openv/netbackup/bin/mail_dr_info.sh‘ script exists on Veritas NetBackup appliances and does not experience the same issue. For UNIX/Linux masters which are not Veritas NetBackup appliances, the script will need to be created and tested.  

The example script below can be used on Linux Master servers and serve as a template for other Unix platforms.
Permissions should be set to 744.

#!/bin/bash
# Copyright (c) 2020 Veritas Technologies LLC. All rights reserved

emailid=$1 # Can be multiple recipients comma separated.
subject=$2 # Subject may contain spaces.
message=$3 # Message file.
attach=$4  # Can be multiple attachment files comma separated.

# Add attachment option if there is one specified.
[ "${attach}" ] && attach="-a ${attach}"

# Adjust mail program argument for multiple attachments.
attach=$(echo ${attach} | sed -e "s|,| -a |g")

/usr/bin/mail -s "${subject}" ${attach} ${emailid} < ${message}

For customers creating a custom mail_dr_info.sh script, the example below splits the files listed in the $4 variable to ensure the disaster recovery files can be actioned by the mail client. 

Example of how to split the files from variable $4 into two separate variables $file1 and $file2 :

file1=$(echo $4 | awk -F, '{print $1}')
file2=$(echo $4 | awk -F, '{print $2}')

The custom mail script must then use $file1 and $file2 in the mail client command syntax to attach both files separately. 

Example:

#!/bin/bash
# Copyright (c) 2020 Veritas Technologies LLC. All rights reserved

emailid=$1 # Can be multiple recipients comma separated.
subject=$2 # Subject may contain spaces.
message=$3 # Message file.
attach=$4  # Can be multiple attachment files comma separated.

file1=$(echo $4 | awk -F, '{print $1}')
file2=$(echo $4 | awk -F, '{print $2}')

# Add attachment option if there is one specified.
[ "${attach}" ] && attach="-a ${attach}"

# Adjust mail program argument for multiple attachments.
attach=$(echo ${attach} | sed -e "s|,| -a |g")

/usr/bin/mail -s "${subject}" ${attach} ${emailid} < ${message}

NOTE: The executable to send email may differ across platforms. Customer's will need to validate the mail client configuration and command syntax of the script before putting the script into production.

Veritas Support are providing this information 'AS IS' in order to assist customers with Catalog DR Email configuration. However, providing customization or debugging of scripts used on customer systems is out of scope for Veritas Support Cases. Customers needing scripting expertise are encouraged to contact Veritas Consulting Services.