Important Update: Cohesity Products Documentation


All Cohesity product documentation are now managed via the Cohesity Docs Portal: https://docs.cohesity.com/HomePage/Content/home.htm. Some documentation available here may not reflect the latest information or may no longer be accessible.

NetBackup™ Troubleshooting Guide

Last Published:
Product(s): NetBackup (11.1)
  1. Introduction
    1.  
      Additional resources on NetBackup logging and status code information
    2.  
      Troubleshooting a problem
    3.  
      Problem report for Technical Support
    4.  
      About gathering information for NetBackup-Java applications
  2. Troubleshooting procedures
    1.  
      About troubleshooting procedures
    2. Troubleshooting NetBackup problems
      1.  
        Verifying that all processes are running on UNIX or Linux servers
      2.  
        Verifying that all processes are running on Windows servers
    3.  
      Troubleshooting installation problems
    4.  
      Troubleshooting configuration problems
    5.  
      Device configuration problem resolution
    6.  
      Testing the primary server and clients
    7.  
      Testing the media server and clients
    8.  
      Resolving network communication problems with UNIX clients
    9.  
      Resolving network communication problems with Windows clients
    10. Troubleshooting vnetd proxy connections
      1.  
        vnetd proxy connection requirements
      2.  
        Where to begin to troubleshoot vnetd proxy connections
      3.  
        Verify that the vnetd process and proxies are active
      4.  
        Verify that the host connections are proxied
      5.  
        Test the vnetd proxy connections
      6.  
        Examine the log files of the connecting and accepting processes
      7.  
        Viewing the vnetd proxy log files
    11. Troubleshooting security certificate revocation
      1.  
        Troubleshooting cloud provider's revoked SSL certificate issues
      2.  
        Troubleshooting cloud provider's CRL download issues
      3.  
        How a host's CRL affects certificate revocation troubleshooting
      4.  
        NetBackup job fails because of revoked certificate or unavailability of CRLs
      5.  
        NetBackup job fails because of apparent network error
      6.  
        NetBackup job fails because of unavailable resource
      7.  
        Primary server security certificate is revoked
      8.  
        Determining a NetBackup host's certificate state
      9.  
        Troubleshooting issues with external CA-signed certificate revocation
    12.  
      About troubleshooting networks and host names
    13. Verifying host name and service entries in NetBackup
      1.  
        Example of host name and service entries on UNIX primary server and client
      2.  
        Example of host name and service entries on UNIX primary server and media server
      3.  
        Example of host name and service entries on UNIX PC clients
      4.  
        Example of host name and service entries on UNIX server that connects to multiple networks
    14.  
      About the bpclntcmd utility
    15.  
      Using the Host properties to access configuration settings
    16.  
      Resolving full disk problems
    17. Frozen media troubleshooting considerations
      1.  
        Logs for troubleshooting frozen media
      2.  
        About the conditions that cause media to freeze
    18. Troubleshooting problems with the NetBackup web services
      1.  
        Viewing NetBackup web services logs
      2.  
        Troubleshooting web service issues after external CA configuration
    19.  
      Troubleshooting problems with the NetBackup web server certificate
    20. Resolving PBX problems
      1.  
        Checking PBX installation
      2.  
        Checking that PBX is running
      3.  
        Checking that PBX is set correctly
      4.  
        Accessing the PBX logs
      5.  
        Troubleshooting PBX security
      6.  
        Determining if the PBX daemon or service is available
    21. Troubleshooting problems with validation of the remote host
      1.  
        Viewing logs pertaining to host validation
      2.  
        Enabling insecure communication with NetBackup 8.0 and earlier hosts
      3.  
        Approving pending host ID-to-host name mappings
      4.  
        Clearing host cache
    22. Troubleshooting Auto Image Replication
      1.  
        Rules for primary servers used with Auto Image Replication (A.I.R.) and SLPs
      2. Targeted A.I.R. trusted primary server operation fails with an external certificate configuration
        1.  
          Troubleshoot adding or updating the trust
        2.  
          Troubleshoot removing the trust
      3.  
        About troubleshooting automatic import jobs that SLP components manage
    23.  
      Troubleshooting network interface card performance
    24.  
      About SERVER entries in the bp.conf file
    25.  
      About unavailable storage unit problems
    26.  
      Resolving a NetBackup Administration operations failure on Windows
    27.  
      Resolving garbled text displayed in NetBackup Administration Console on a UNIX computer
    28.  
      Troubleshooting error messages in the NetBackup web UI and the NetBackup Administration Console
    29.  
      Extra disk space required for logs and temporary files for the NetBackup Administration Console
    30.  
      Unable to logon to the NetBackup Administration Console after external CA configuration
    31.  
      Troubleshooting file-based external certificate issues
    32.  
      Troubleshooting issues with external certificate configuration
    33.  
      Troubleshooting Windows certificate store issues
    34.  
      Troubleshooting backup failures
    35.  
      Troubleshooting backup failure issues with NAT clients or NAT servers
    36.  
      Troubleshooting issues with the NetBackup Messaging Broker (or nbmqbroker) service
    37.  
      Troubleshooting issues with email notifications for Windows systems
    38.  
      Troubleshooting issues with KMS configuration
    39.  
      Troubleshooting issues with initiating the NetBackup CA migration because of large key size
    40.  
      Troubleshooting issues with the non-privileged user (service user) account
    41.  
      Troubleshooting issues with group name format in the auth.conf file
    42.  
      Troubleshooting the VxUpdate add package process
    43.  
      Troubleshooting issues with FIPS mode
    44.  
      Troubleshooting issues with malware scanning
    45.  
      Troubleshooting issues with NetBackup jobs that are enabled for data-in-transit encryption
    46.  
      Troubleshooting issues with Unstructured Data Instant Access
    47.  
      Troubleshooting issues with multifactor authentication
    48.  
      Troubleshooting issues with multi-person authorization
    49.  
      Troubleshooting connections to the NetBackup Scale-Out Relational Database
    50.  
      Troubleshooting issues with private key encryption
    51.  
      Troubleshooting issues with the security configuration risk feature
    52.  
      Troubleshooting issues with the risk engine-based anomaly detection options
    53. Troubleshooting NetBackup WebSocket server connection
      1.  
        Problems validating the endpoint server in the WebSocket Server dialog
      2.  
        Problems saving the NetBackup endpoint credentials in the WebSocket Server dialog
      3.  
        Problems deleting the WebSocket server endpoint from NetBackup
      4.  
        Problems displaying the list of WebSocket servers that were added in NetBackup
      5.  
        Problems activating or deactivating the endpoint server
      6.  
        Additional NBWSS issues
    54.  
      Troubleshooting issues with the network access control feature
    55.  
      Troubleshooting issues with freeze mode
    56.  
      Troubleshooting issues with rotation of external CA-issued certificates
  3. Using NetBackup utilities
    1.  
      About NetBackup troubleshooting utilities
    2.  
      About the analysis utilities for NetBackup debug logs
    3.  
      About the Log collection utility
    4.  
      About network troubleshooting utilities
    5. About the NetBackup support utility (nbsu)
      1.  
        Output from the NetBackup support utility (nbsu)
      2.  
        Example of a progress display for the NetBackup support utility (nbsu)
    6. About the NetBackup consistency check utility (NBCC)
      1.  
        Output from the NetBackup consistency check utility (NBCC)
      2.  
        Example of an NBCC progress display
    7.  
      About the NetBackup consistency check repair (NBCCR) utility
    8.  
      About the nbcplogs utility
    9. About the robotic test utilities
      1.  
        Robotic tests on UNIX
      2.  
        Robotic tests on Windows
    10. About the NetBackup Smart Diagnosis (nbsmartdiag) utility
      1.  
        Workflow to use the nbsmartdiag utility for NetBackup host communication
    11.  
      About log collection by job ID
  4. Disaster recovery
    1.  
      About disaster recovery
    2.  
      Recommended backup practices
    3.  
      Requirements and notes for disaster recovery
    4.  
      Disaster recovery packages
    5.  
      About disaster recovery settings
    6. About disk recovery procedures for UNIX and Linux
      1. About recovering the primary server disk on Linux
        1.  
          Recovering the primary server when root is intact
        2.  
          Recovering the primary server when the root partition is lost
      2.  
        About recovering the NetBackup media server disk for UNIX
      3.  
        Recovering the system disk on a UNIX client workstation
    7. About clustered NetBackup server recovery for UNIX and Linux
      1.  
        Replacing a failed node on a UNIX or Linux cluster
      2.  
        Recovering the entire UNIX or Linux cluster
    8. About disk recovery procedures for Windows
      1. About recovering the primary server disk for Windows
        1.  
          Recovering the primary server with Windows intact
        2.  
          Recovering the primary server and Windows
      2.  
        About recovering the NetBackup media server disk for Windows
      3.  
        Recovering a Windows client disk
    9. About clustered NetBackup server recovery for Windows
      1.  
        Replacing a failed node on a Windows VCS cluster
      2.  
        Recovering the shared disk on a Windows VCS cluster
      3.  
        Recovering the entire Windows VCS cluster
    10.  
      Generating a certificate on a clustered primary server after disaster recovery installation
    11.  
      About the DR_PKG_MARKER_FILE environment variable
    12.  
      Restoring the disaster recovery package on Windows
    13.  
      Restoring the disaster recovery package on Linux
    14. Options to recover the NetBackup catalog
      1. Prerequisites for recovering the NetBackup catalog or NetBackup catalog image files
        1.  
          Establishing a connection with NAT media server before catalog recovery
      2.  
        About NetBackup catalog recovery on Windows computers
      3.  
        About NetBackup catalog recovery from disk devices
      4.  
        About NetBackup catalog recovery and symbolic links
      5.  
        NetBackup disaster recovery email example
      6. About recovering the entire NetBackup catalog
        1.  
          Recovering the entire NetBackup catalog using the NetBackup catalog recovery wizard
        2.  
          Recovering the entire NetBackup catalog using bprecover -wizard
        3.  
          Specifying the NetBackup job ID number after a catalog recovery
      7. About recovering the NetBackup catalog image files
        1.  
          Recovering the NetBackup catalog image files using the NetBackup catalog recovery wizard
        2.  
          Recovering the NetBackup catalog image files using bprecover -wizard
      8. About recovering the NetBackup databases
        1.  
          Recovering the NetBackup database from a backup
        2.  
          Recovering the NetBackup database from staging
        3.  
          About processing the NetBackup database in staging
        4.  
          Terminating database connections
      9.  
        Recovering the NetBackup catalog when NetBackup Access Control is configured
      10.  
        Recovering the NetBackup catalog from a nonprimary copy of a catalog backup
      11.  
        Recovering the NetBackup catalog without the disaster recovery file
      12.  
        Recovering a NetBackup user-directed online catalog backup from the command line
      13.  
        Restoring files from a NetBackup online catalog backup
      14.  
        Unfreezing the NetBackup online catalog recovery media
      15.  
        Steps to carry out when you see exit status 5988 during catalog recovery

Troubleshooting issues with rotation of external CA-issued certificates

This topic provides troubleshooting information about the issues that are specific to rotation of external CA-issued certificates.

For more information on freeze mode, see the NetBackup Security and Encryption Guide.

Table:

Sr. No.

Issue

Possible reason

Resolution

1

Exception with message:

"Cannot connect nbsl"

Web service logs show the following log statement:

Cannot retrieve hostName from system property

NetBackup Service Layer service might not be running. CLIENT_NAME or SERVER in bp.conf is not correct for the given primary server.

Check if the NetBackup Service Layer (NBSL) service is up and running.

Later, increase verbosity, and retry the operations.

Contact Cohesity technical support if the issue still persists.

2

Exception with error code 8752:

"The requested operation is not supported for the NetBackup version of the remote host."

Web service logs show the following log statement:

ECA automatic host cert rotation is not allowed on FLEX-SCALE

The host for which certificates are being uploaded is Flex Scale deployment.

The API is not supported for Flex Scale deployment.

Use the Flex Scale-specific methods to configure external CA-issued certificates.

Web service logs show the following log statement:

ECA automatic host cert rotation is not allowed on Cloudscale.

The host for which certificates are being uploaded is a Cloud Scale deployment.

External CA-issued certificates are not supported for host communication on Cloud Scale deployment, therefore the rotation of ECA certificates cannot be configured on a Cloud Scale setup.

3

Exception with message:

Invalid operation in the API request body. The operation is not supported or disabled.

Web service logs show the following log statement:

isValidNBUVersion : 0

The host for which certificates are being uploaded is earlier than NetBackup 11.0

This API is not supported for earlier versions.

Renew the certificates manually for hosts earlier than 11.0.

4

Uploading ECA artifacts take 15-20 seconds and fails for NetBackup servers.

Web service logs show the following log statement:

Received exception while getting Container deployment type

If services of the host are down (or the bprd service is not running), only PEM files can be uploaded.

For example: In case of clustered primary server, on an inactive node, the bprd process is not running. Therefore, only PEM files can be uploaded.

In a case where non-PEM files are uploaded, NetBackup needs to connect to the host to check if the files are supported or not.

This is valid for NetBackup servers only.

Start all the NetBackup services on the host and retry the upload operation.

Ensure that the bprd service is also running. Else use PEM files as they are supported across all the types of deployments.

5

Renewal failed with error:

Failed to retrieve external certificate artifacts from credential management system.

Web service logs show the following log statement:

Failed to fetch eca artifacts CMS credentials

Credentials management or database service may not be running.

Ensure that the credentials management and database services are up and running. Later, increase the verbosity.

Contact Cohesity technical support if the issue still persists.

6

Renewal failed with error:

Failed to process the data of external certificate artifacts during download.

Web service logs show the following log statement:

Failed to decode eca artifacts CMS credentials

Primary server has not sent base64 encoded data to the respective host.

Check if the NetBackup Service Layer (NBSL) service is up and running. Later, increase the verbosity.

Contact Cohesity technical support if the issue still persists.

7

Renewal failed with error:

Failed to save external certificate artifacts to the NetBackup temporary location.

Web service logs show the following log statement:

Writing ECA host artifacts operation failed

Unable to write uploaded artifacts at the NetBackup temporary location: Install_PATH/tmp

Ensure that the NetBackup services have the write permissions on the NetBackup temporary location Install_PATH/tmp

8

Renewal failed with error:

Failed to validate external certificate enrollment

Web service logs show the following log statement:

ECA host certificate enrollment dry run failed Check Web Service logs as well.

Dry run of uploaded artifacts failed. Possible reasons:

  • The uploaded ECA artifacts have the private key size 1024.

  • CRL checks failed for the certificate.

  • Ensure that you use valid artifacts.

    In case of FIPS, upload private key size >= 2048.

  • For CRL issue, ensure that you to upload valid certificates.

9

Renewal failed with error:

Failed to validate external certificate enrollment.

Web service logs show the following log statement:

Failed to perform enroll certificate, with error code : 44

The size of the certificate chain is more than 40 KB around.

Ensure that the certificate chain is not too big in size and is less than 40 KB.

10

Renewal failed with error:

Failed to save external certificate artifacts to the NetBackup default location.

Web service logs show the following log statement:

Updating ECA host artifacts at final location operation got failed

Unable to write uploaded artifacts at the NetBackup-managed ECA artifacts location:

Ensure that NetBackup services have the 'write' permissions on the following NetBackup-managed ECA artifacts locations:

Install_PATH/var/vxss/credentials/ecaartifacts

For cacert:

Install_PATH/var/vxss/

11

Renewal failed with error:

Failed to update the paths of external certificate artifacts in the NetBackup configuration files.

Web service logs show the following log statement:

New artifacts path update failed

Unable to update the NetBackup configuration files.

For UNIX: bp.conf file

For Windows: Registry

Check if NetBackup Service Layer (NBSL) service is up and running. Later, increase the verbosity.

Contact Cohesity technical support if the issue still persists.

12

Renewal process is blocked at the validation phase.

It is possible that the CRL URLs are not accessible from the host and CRL check level was defined as LEAF/CHAIN while uploading certificates.

Ensure that the CRL URLs are accessible from the host. Else use the commands to set the CRL check level to DISABLE in the bp.conf configuration file.

13

After certificate rotation backup or backup from snapshot jobs are failing with error code: 5982

The host is not able to verify connections using CRL. Possibly because the CRL URLs are not accessible from the host.

Ensure that the CRL URLs are accessible from the host. Else use the commands to set the CRL check level to DISABLE in the bp.conf configuration file.