The NBDB database transaction log (NBDB.log) may become too large or corrupt for proper NetBackup operation.

Problem

Problem occurs when NetBackup Relational Database Manager service failed to start. When running following command shows Database unavailable.

Error Message

/usr/openv/db/bin/nbdb_ping
Database [NBDB] is not available.

There may be errors reported by the ASA database engine in the /usr/openv/db/log/server.log file as well. These errors would look similar to the following:

Error: Cannot open transaction log file -- Can't use log file "/usr/openv/db/data/NBDB.log" since it is shorter than expected

Cause

Problem occurs when transaction log for the ASA database that runs under NetBackup becomes corrupt or too large for proper operation. For example, in a file system that limits file size to 2GB, the transaction log will be truncated and corrupt once it reaches the 2Gb limit. The transaction log is truncated during online catalog backup, but it might be possible that catalog backups did not take place to prevent the log from growing too large.

The ASA database failing due to transaction log problems can include these errors:

• Catalog backups and regular backups can hang.
• Operator is unable to cancel queued jobs.
• The EMM database does not come back up after restarting NetBackup

Solution

Rebuild the transaction log via the following steps:

1. Stop NetBackup if it is not already stopped:
   /usr/openv/netbackup/bin/goodies/netbackup stop
2. To add the necessary environment variables to your shell, run the command:
   source /usr/openv/db/vxdbms_env.sh
   • Note 1: If using 'csh' for a shell, the command is 'source /usr/openv/db/vxdbms_env.csh' without the quotes.
   • Note 2: Depending on the root user's shell environment, this command could fail with a "Parameter not set" or "unbound variable" error. If one of these errors occurs, run the 'set +u' command and then reattempt the '. /usr/openv/db/vxdbms_env.sh' command.
3. Change directories to where the NBDB resides:
   cd /usr/openv/db/data
4. Remove or rename the bad transaction log:
   mv NBDB.log NBDB.log.bad
   • If BMR is used, also rename the BMR transaction log:
     mv BMRDB.log BMRDB.log.bad
5. To force database recovery, run the command:
   /usr/openv/db/bin/dbsrv9 -f /usr/openv/db/data/NBDB.db
   • If BMR is used, force recovery of the BMR database:
     /usr/openv/db/bin/dbsrv9 -f /usr/openv/db/data/BMRDB.db
   • Note 3: in the above command replace dbsrv9 by dbsrv11, dbsrv12, dbsrv16 or dbsrv17 depending on the installed version version of NetBackup.
   • Note 4: If the command 'dbsrv11 -f /usr/openv/db/data/NBDB.db' should fail with the error: "A database server with that name has already started ", please check /usr/openv/tmp and rename the sqlany directory, and all of the sqlany files. Restart recovery as per step 5 above and the 'dbsrv11 -f /usr/openv/db/data/NBDB.db' should work.
   • Note 5: The NBDB.log will not be created until any transaction actually occurs, such as NetBackup making connection to the EMM DB:
6. Stop the database by running the command:
   /usr/openv/db/bin/nbdbms_start_server -stop
7. To start the database, run the command:
   /usr/openv/db/bin/nbdbms_start_server
8. To test if the database is operational, run the command below:
   /usr/openv/db/bin/nbdb_ping
   And the output, should return something similar to the following:
   Database [NBDB] is alive and well on server [NB_master1].