NetBackup jobs responsible for Oracle metadata collection fails if SQLNET.AUTHENTICATION_SERVICES=NONE

Problem

The jobs that transfer the backup pieces for the Oracle database are successful. However, the jobs responsible for metadata collection report a failure if the Oracle database has SQLNET.AUTHENTICATION_SERVICES=NONE set in the Oracle sqlnet.ora file.

This prevents NetBackup from retrieving the metadata about the backup and storing it into the DARS database.

The backup images are complete and restorable, just not available for Guided Recovery.

Error

If the backup is either scheduled by NetBackup or is run through an Oracle Intelligent Policy, the child jobs complete with status 0, but the parent jobs are partially successful (1).

Activity Monitor for server-initiated backup

The Job Details for the server-initiated backup confirms the backup completed successfully, but then a user privilege conflict occurred when connecting to the Oracle instance to gather the metadata.

...snip...

8/22/2014 2:27:12 PM - Info bphdb(pid=327260) INF - channel ch00: backup set complete
8/22/2014 2:27:13 PM - Info bprd(pid=328640) Storing the results of the backup in the DARS database
8/22/2014 2:27:13 PM - Info bprd(pid=328640) Results for: ORAC112
8/22/2014 2:27:13 PM - Info bprd(pid=328640) Backup ID: client_1408735626
8/22/2014 2:27:13 PM - Error bprd(pid=328640) Failed sending the results of the backup to DARS: ERR - Unable to connect to ORAC112. ORA-01031: insufficient privileges. Either an invalid username or password was entered in an attempt to log on to Oracle or the user does not have SYSDBA privileges.
8/22/2014 2:27:13 PM - Error bprd(pid=328640) Failed to store the results of the backup in the DARS database 8/22/2014 2:27:14 PM - Info bphdb(pid=327260) INF - released channel: ch00

...snip...

8/22/2014 2:27:52 PM - Info bphdb(pid=327260) INF - Recovery Manager complete.
8/22/2014 2:27:52 PM - Info bphdb(pid=327260) INF - End of Recovery Manager output.
8/22/2014 2:27:52 PM - Info bphdb(pid=327260) INF - End Oracle Recovery Manager.
8/22/2014 2:27:53 PM - Warning bpbrm(pid=327472) from client stardrive: WRN - Metadata cataloging for a child job failed
8/22/2014 2:27:53 PM - Error bpbrm(pid=327472) from client stardrive: ERR - bphdb exit status = x: the requested operation was partially successful
8/22/2014 2:27:53 PM - Info bpbrm(pid=327472) validating image for client <client_name>
8/22/2014 2:27:54 PM - Info bphdb(pid=327260) done. status: 1: the requested operation was partially successful
8/22/2014 2:27:54 PM - end writing
the requested operation was partially successful (1)

The job was successfully completed, but some files may have been busy or inaccessible. See the problems report or the client's logs for more details. 

If the backup is not scheduled through NetBackup, the application backup jobs complete with status 0, but the DARS_DATABASE_UPDATE job fails with status 26.

Activity Monitor for client-initiated backup

The Job Details for the client-initiated backup show a similar failure.

8/22/2014 9:17:46 AM - Info bprd(pid=326908) Storing the results of the backup in the DARS database
8/22/2014 9:17:46 AM - Info bprd(pid=326908) Results for: orac112
8/22/2014 9:17:46 AM - Info bprd(pid=326908) Backup ID: client_1408717058
8/22/2014 9:17:46 AM - Info bprd(pid=326908) Backup ID: client_1408716993
8/22/2014 9:17:46 AM - Error bprd(pid=326908) Failed sending the results of the backup to DARS: ERR - Unable to connect to ORAC112. ORA-01031: insufficient privileges. Either an invalid username or password was entered in an attempt to log on to Oracle or the user does not have SYSDBA privileges.
8/22/2014 9:17:46 AM - Error bprd(pid=326908) Failed to store the results of the backup in the DARS database

The dbclient debug log shows that the metadata collection during the sbtend() processing failed with status 160.

14:38:11.701 [2492.289712] <16> int_RunWinCmd: ERR - Child terminated with error: 160
14:38:11.701 [2492.289712] <16> int_PerformMetadataCataloging: ERR - Failed to run metadata catalog command
14:38:11.701 [2492.289712] <2> xbsa_SendChildJobStatus: INF - entering
14:38:11.701 [2492.289712] <4> VxBSASendChildJobStatus: INF - entering VxBSASendChildJobStatus
14:38:11.701 [2492.289712] <4> VxBSASendChildJobStatus: Metadata cataloging for this job failed, creating file <C:\Program Files\Veritas\NetBackup\Logs\user_ops\dbext\logs\failed_metadata_cataloging_jobs.177>.
14:38:11.702 [2492.289712] <2> xbsa_SendChildJobStatus: INF - leaving (0)

Note: Because the actual backup is successful, NetBackup returns status 0 to RMAN so that it will retain and catalog the information about the backup pieces.  RMAN will not report any failures. 

Cause

The NetBackup DARS metadata collection was designed to use only operating system (OS) authentication when collecting the metadata for the Oracle backup that has just completed. If SQLNET.AUTHENTICATION_SERVICES=NONE is configured, then Oracle requires an Oracle username and password for all connections to the database.

Solution

If permitted by the database environment, set SQLNET.AUTHENTICATION_SERVICES=NTS to allow metadata collection.
If the database environment does not permit setting SQLNET.AUTHENTICATION_SERVICES=NTS, then disable metadata collection to allow the backup to complete successfully.

Note: NetBackup Guided Recovery cannot be used without prior metadata collection.

The following methods can be used to control whether metadata collection is enabled or disabled for either all backup jobs or for specific backup jobs.

1. To set the default for all jobs on the client (myOracleClient), send this configuration setting to the client from the master or media server.

server$ echo ORACLE_METADATA=NO | bpsetconfig -h myOracleClient

Note: The bpsetconfig command is located in the install_path/netbackup/bin/admincmd directory.

2. To override the default for a specific job, use the RMAN SEND command in the backup script.

ALLOCATE CHANNEL … ;
SEND ’NB_ORA_METADATA=NO’;
BACKUP … ;

Note: Either setting can be changed to ‘YES’ if some database instances on the client host are configured for OS authentication.