Veritas NetBackup™ Troubleshooting Guide

Last Published:
Product(s): NetBackup (8.2)
  1. Introduction
    1.  
      NetBackup logging and status code information
    2.  
      Troubleshooting a problem
    3.  
      Problem report for Technical Support
    4.  
      About gathering information for NetBackup-Java applications
  2. Troubleshooting procedures
    1.  
      About troubleshooting procedures
    2. Troubleshooting NetBackup problems
      1.  
        Verifying that all processes are running on UNIX servers
      2.  
        Verifying that all processes are running on Windows servers
    3.  
      Troubleshooting installation problems
    4.  
      Troubleshooting configuration problems
    5.  
      Device configuration problem resolution
    6.  
      Testing the master server and clients
    7.  
      Testing the media server and clients
    8.  
      Resolving network communication problems with UNIX clients
    9.  
      Resolving network communication problems with Windows clients
    10. Troubleshooting vnetd proxy connections
      1.  
        vnetd proxy connection requirements
      2.  
        Where to begin to troubleshoot vnetd proxy connections
      3.  
        Verify that the vnetd process and proxies are active
      4.  
        Verify that the host connections are proxied
      5.  
        Test the vnetd proxy connections
      6.  
        Examine the log files of the connecting and accepting processes
      7.  
        Viewing the vnetd proxy log files
    11. Troubleshooting security certificate revocation
      1.  
        Troubleshooting cloud provider's revoked SSL certificate issues
      2.  
        Troubleshooting cloud provider's CRL download issues
      3.  
        How a host's CRL affects certificate revocation troubleshooting
      4.  
        NetBackup job fails because of revoked certificate or unavailability of CRLs
      5.  
        NetBackup job fails because of apparent network error
      6.  
        NetBackup job fails because of unavailable resource
      7.  
        Master server security certificate is revoked
      8.  
        Determining a NetBackup host's certificate state
      9.  
        Troubleshooting issues with external CA-signed certificate revocation
    12.  
      About troubleshooting networks and host names
    13. Verifying host name and service entries in NetBackup
      1.  
        Example of host name and service entries on UNIX master server and client
      2.  
        Example of host name and service entries on UNIX master server and media server
      3.  
        Example of host name and service entries on UNIX PC clients
      4.  
        Example of host name and service entries on UNIX server that connects to multiple networks
    14.  
      About the bpclntcmd utility
    15.  
      Using the Host Properties window to access configuration settings
    16.  
      Resolving full disk problems
    17. Frozen media troubleshooting considerations
      1.  
        Logs for troubleshooting frozen media
      2.  
        About the conditions that cause media to freeze
    18. Troubleshooting problems with the NetBackup web services
      1.  
        Viewing NetBackup web services logs
      2.  
        Troubleshooting web service issues after external CA configuration
    19.  
      Troubleshooting problems with the NetBackup web server certificate
    20. Resolving PBX problems
      1.  
        Checking PBX installation
      2.  
        Checking that PBX is running
      3.  
        Checking that PBX is set correctly
      4.  
        Accessing the PBX logs
      5.  
        Troubleshooting PBX security
      6.  
        Determining if the PBX daemon or service is available
    21. Troubleshooting problems with validation of the remote host
      1.  
        Viewing logs pertaining to host validation
      2.  
        Enabling insecure communication with NetBackup 8.0 and earlier hosts
      3.  
        Approving pending host ID-to-host name mappings
      4.  
        Clearing host cache
    22. About troubleshooting Auto Image Replication
      1. Troubleshooting Auto Image Replication
        1.  
          Targeted AIR trusted master server operation failed in case of external certificate configuration
      2.  
        About troubleshooting automatic import jobs
    23.  
      Troubleshooting network interface card performance
    24.  
      About SERVER entries in the bp.conf file
    25.  
      About unavailable storage unit problems
    26.  
      Resolving a NetBackup Administration operations failure on Windows
    27.  
      Resolving garbled text displayed in NetBackup Administration Console on a UNIX computer
    28.  
      Unable to logon to the NetBackup Administration Console after external CA configuration
    29.  
      Troubleshooting file-based external certificate issues
    30.  
      Troubleshooting Windows certificate store issues
    31.  
      Troubleshooting backup failures
    32.  
      Troubleshooting backup failure issues with NAT clients
    33.  
      Troubleshooting issues with the NetBackup Messaging Broker (or nbmqbroker) service
  3. Using NetBackup utilities
    1.  
      About NetBackup troubleshooting utilities
    2.  
      About the analysis utilities for NetBackup debug logs
    3.  
      About the Logging Assistant
    4.  
      About network troubleshooting utilities
    5. About the NetBackup support utility (nbsu)
      1.  
        Output from the NetBackup support utility (nbsu)
      2.  
        Example of a progress display for the NetBackup support utility (nbsu)
    6. About the NetBackup consistency check utility (NBCC)
      1.  
        Output from the NetBackup consistency check utility (NBCC)
      2.  
        Example of an NBCC progress display
    7.  
      About the NetBackup consistency check repair (NBCCR) utility
    8.  
      About the nbcplogs utility
    9. About the robotic test utilities
      1.  
        Robotic tests on UNIX
      2.  
        Robotic tests on Windows
  4. Disaster recovery
    1.  
      About disaster recovery
    2.  
      About disaster recovery requirements
    3.  
      Disaster recovery packages
    4.  
      About disaster recovery settings
    5.  
      Recommended backup practices
    6. About disk recovery procedures for UNIX and Linux
      1. About recovering the master server disk for UNIX and Linux
        1.  
          Recovering the master server when root is intact
        2.  
          Recovering the master server when the root partition is lost
      2.  
        About recovering the NetBackup media server disk for UNIX
      3.  
        Recovering the system disk on a UNIX client workstation
    7. About clustered NetBackup server recovery for UNIX and Linux
      1.  
        Replacing a failed node on a UNIX or Linux cluster
      2.  
        Recovering the entire UNIX or Linux cluster
    8. About disk recovery procedures for Windows
      1. About recovering the master server disk for Windows
        1.  
          Recovering the master server with Windows intact
        2.  
          Recovering the master server and Windows
      2.  
        About recovering the NetBackup media server disk for Windows
      3.  
        Recovering a Windows client disk
    9. About clustered NetBackup server recovery for Windows
      1.  
        Replacing a failed node on a Windows VCS cluster
      2.  
        Recovering the shared disk on a Windows VCS cluster
      3.  
        Recovering the entire Windows VCS cluster
    10.  
      Generating a certificate on a clustered master server after disaster recovery installation
    11.  
      About restoring disaster recovery package
    12.  
      About the DR_PKG_MARKER_FILE environment variable
    13.  
      Restoring disaster recovery package on Windows
    14.  
      Restoring disaster recovery package on UNIX
    15. About recovering the NetBackup catalog
      1.  
        About NetBackup catalog recovery on Windows computers
      2.  
        About NetBackup catalog recovery from disk devices
      3.  
        About NetBackup catalog recovery and symbolic links
      4. About NetBackup catalog recovery and OpsCenter
        1.  
          Specifying the NetBackup job ID number after a catalog recovery
      5.  
        NetBackup disaster recovery email example
      6. About recovering the entire NetBackup catalog
        1.  
          Recovering the entire NetBackup catalog using the Catalog Recovery Wizard
        2.  
          Recovering the entire NetBackup catalog using bprecover -wizard
      7. About recovering the NetBackup catalog image files
        1.  
          Recovering the NetBackup catalog image files using the Catalog Recovery Wizard
        2.  
          Recovering the NetBackup catalog image files using bprecover -wizard
      8. About recovering the NetBackup relational database
        1.  
          Recovering NetBackup relational database files from a backup
        2.  
          Recovering the NetBackup relational database files from staging
        3.  
          About processing the relational database in staging
      9.  
        Recovering the NetBackup catalog when NetBackup Access Control is configured
      10.  
        Recovering the NetBackup catalog from a nonprimary copy of a catalog backup
      11.  
        Recovering the NetBackup catalog without the disaster recovery file
      12.  
        Recovering a NetBackup user-directed online catalog backup from the command line
      13.  
        Restoring files from a NetBackup online catalog backup
      14.  
        Unfreezing the NetBackup online catalog recovery media
      15.  
        Steps to carry out when you see exit status 5988 during catalog recovery

Unable to logon to the NetBackup Administration Console after external CA configuration

Review the troubleshooting following scenarios.

For information on the external CA support in NetBackup, refer to the NetBackup Security and Encryption Guide.

Scenario

If the vnetd service is down on the host to which the NetBackup Administration Console is connecting

Recommended action

Check if the services are up on the host and try logging in again.

Scenario

If external certificate's private key is not available or is in an incorrect format, error VRTS-28678 is displayed.

Recommended action
  • Check if the path provided for the ECA_PRIVATE_KEY_PATH configuration option is valid (it should not be empty).

  • Check if the path provided for ECA_PRIVATE_KEY_PATH is accessible and also if the private key file has required access permissions.

  • Provide a valid private key and try logging in again.

In case of Windows certificate store, do the following:

  • Run the certlm.msc command.

    In case certlm.msc is not working, you can access the Windows certificate store by running the mmc.exe command. Go to File > Add Remove Snap in.

  • Open the certificate by double clicking it.

    The certificate with private key should have a message stating that you have a private key corresponding to this certificate.

Scenario

If the external certificate is not present while you establish the trust with the NetBackup Administration Console.

Recommended action
  • Check if the path provided for the ECA_TRUST_STORE_PATH configuration option is not empty.

  • Check if the path provided for ECA_TRUST_STORE_PATH is accessible and also if the CA certificate file has required access permissions.

  • Provide a valid external certificate and try logging in.

In case of Windows certificate store, do the following:

  • Check if the root CA certificate is added in the Windows Cert Store's Trusted Root Certificate Authorities.

  • Run certlm.msc command. In the certificate management window, open the store named Trusted Root Certificate Authorities. The Trusted Root Certificate Authorities store contains all the self-signed certificates that are trusted by that machine.

    In case certlm.msc is not working, you can access the Windows certificate store by running mmc.exe. Go to File > Add Remove Snap in.

    • Select certificates from left hand side.

    • Click Add.

    • Select computer account. Click Next.

    • Click Finish and then OK.

    • Click Trusted Root Certification Authorities > Certificates.

    • Check if the root CA certificate in the certificate chain is present in the Trusted Root Certificate Authorities store.

  • If the root CA certificate is not present, do the following:

    • Click All Actions > Import.

    • Select .PEM or .CRT or .CER file of the certificate and click Import.

    Note:

    All the certificates should be imported in the local machine store and not in the current user store. You can verify the current store in the certificate management window.

  • Add a valid external CA certificate and try logging in.

Scenario

If an external CA-signed certificate is not present or not accessible, the following error is displayed:

The host does not have external CA-signed certificate. The certificate is mandatory to establish a secure connection.

Recommended action
  • Check if the path provided for ECA_CERT_PATH in NetBackup configuration file is not empty.

  • Check if the path provided for ECA_CERT_PATH points to the entire certificate chain.

  • Check if the path provided for ECA_CERT_PATH is accessible and also if it has required access permissions.

  • Provide a valid external CA-signed certificate and try logging in.

In case of Windows certificate store, do the following:

  • Check if ECA_CERT_PATH contains the appropriate value: Windows Certificate Store Name\Issuer Name\Subject Name. Verify if the certificate exists in the Windows certificate store.

    • Run the certlm.msc command.

      In case certlm.msc is not working, you can access the Windows certificate store by running the mmc.exe.

      File > Add Remove Snap in.

    • Navigate to your certificate as per your input Windows Certificate Store Name\Issuer Name\Subject Name.

    • Open your certificate by double-clicking it.

    • Ensure that it is valid, has a private key, a correct issuer name, and a correct subject name.

      If you are using $hostname in Subject name, check that certificate subject has fully qualified domain name of the host.

      If this is not the case, either change the ECA_CERT_PATH or put the right certificate in Windows certificate store and then try logging in.

Scenario

Certificate revocation list (CRL) is not signed by a trusted authority.

Recommended action

This may occur at the time of login if the master server was configured to use NetBackup certificates and later it was enabled to use external certificates and vice versa. So the NetBackup Administration Console starts using the new CRL if you click Activity Monitor, locks the screen, tries to login again or in the periodic checks after every 1 hour, the certificate revocation status verification fails.

To fix this issue, you need to close the console and login again so that the peer host's certificate and the CRL are in sync.

If logging in again does not fix the issue then the reason can be the new CRL was not downloaded.

Run following command after correcting the CRL format:

UNIX: /usr/openv/netbackup/bin/nbcertcmd -updateCRLCache

Windows: install_path\Veritas\Netbackup\bin\nbcertcmd -updateCRLCache

Scenario

The revocation status of the host certificate cannot be verified using the CRL, because the CRL format is not valid.

Recommended action

This error can occur if a delta CRL is used.

NetBackup does not support delta CRLs, so you need to use non-delta CRLs.

Run following command after correcting the CRL format:

UNIX: /usr/openv/netbackup/bin/nbcertcmd -updateCRLCache

Windows: install_path\Veritas\Netbackup\bin\nbcertcmd -updateCRLCache

Scenario

The certificate of the host name is revoked.

Recommended action

If the certificate was revoked in error, reissue a certificate for the host.

If the certificate was revoked intentionally, a security breach may have occurred. Contact your security administrator.

Scenario

The Certificate Revocation List could not be downloaded. Therefore the certificate revocation status could not be verified.

Recommended action

The possible causes include the following:

  • ECA_CRL_PATH is missing or has incorrect path.

  • The CRL file is missing. The CRL file is corrupted.

  • The CRL file could not be locked.

  • The CRL file could not be unlocked.

For more information, see the bpjava logs.

Scenario

The Certificate Revocation List is not updated. Therefore the certificate revocation status could not be verified.

Recommended action

The possible causes include the following:

  • The next update date / time of the CRL is older than the current system date / time.

  • The CRL was valid at the time of login. The console was open and now the CRL has become invalid.

Ensure that the system time is correct.

In case the new CRL was not downloaded, run the following command

UNIX: /usr/openv/netbackup/bin/nbcertcmd -updateCRLCache

Windows: install_path\Veritas\Netbackup\bin\nbcertcmd -updateCRLCache

Scenario

Unable to connect to the NetBackup Web Management Console service.

Recommended action

The possible causes include the following:

  • The NetBackup Web management Console service is down.

  • ECA_CERT_PATH does not point to the entire certificate chain.

  • Web service certificate's issuer and the issuer of the host certificate may not match.

    If both the certificates are not issued by the same external CA, certificate trust verification fails.

Review the following:

  • It is mandatory to provide the path to the certificate file that contains the entire chain of certificates (except the root certificate).

  • If chain is not specified, the certificate trust verification fails and the console is not able to connect to the web service.

  • Ensure that the web server's certificate and the host certificate are issued by same external CA.