Tomcat and nbwebsvc certificates are not renewed automatically on the master server with non-English locale.

Article: 100044601
Last Published: 2022-07-28
Ratings: 18 3
Product(s): NetBackup & Alta Data Protection

Problem

Tomcat and nbwebsvc certificates are not renewed automatically on the master server with non-English locale.
Tomcat and nbwebsvc certificates expire one year after their issue date. They are supposed to be automatically renewed 180 days before the expiration date.
But they are not renewed automatically if non-English local is used for the master server.
The issue affects NetBackup 8.0 through 8.1.2 on all of the operating system using non-English locale.

For example, you may experience the following situation when a certificate expires.

  • The expiration of these certificates results in failures of all of the NetBackup operations such as running backups, logging in to NetBackup Administration Console.
  • You cannot login to NetBackup Administration Console, but the backup jobs will continue to run as normal after the expiration of the certificates

 

Error Message

The renewal process is performed every 24 hours, and nbwebservice log may show the following messages (java.text.ParseException: Unparseable date) each time.
You may find these messages in OID 466 (NetBackup 8.1 and earlier) or OID 495 (NetBackup version 8.1.1 and later).

NBCertRenewTask failed to renew nbwebsvc user credentials - java.text.ParseException: Unparseable date
NBCertRenewTask failed to renew web service NBAC credentials - java.text.ParseException: Unparseable date
NBCertRenewTask failed to renew TOMCAT credentials - java.text.ParseException: Unparseable date

When certificate expires, the following errors appear depending on the situation.

A backup job fails with Status 8506: The certificate has expired.
NetBackup Administration Console fails to login to the Master Server with Status 7656: Certificate Revocation List is out of date.
"nbcertcmd -getCertificate -force" fails with Status 8625: Server is unavailable to process the request. Please try later.

 

Cause

The renewal process on non-English locale fails to parse expiry date. When NBCertRenew class of renewal process does format and parse dates with Java SimpleDateFormat class, an exception occurs because it does not consider locale.

To check local language is set to English or non-English:

On Appliance/Linux run command:

#locale

On Windows Master Server:

  1. Click Start, then Control Panel
  2. Click Clock, Language and Region
  3. Click Region.

 

Solution

Workaround on Master Server:

Every NetBackup Master Server version 8.0 to 8.1.2 is affected by this issue, if its OS is non-English locale. Without installing the hotfix EEB, the certificate will be valid about a year since the installation of the Master Server. 
If the certificate has already expired, it is mandatory to perform this workaround before applying the hotfix EEB.

  • To prevent a future occurrence of this issue, the hotfix EEB must be installed on the Master Server.
  • If the certificate is still valid, applying the hotfix EEB will avoid this issue happening in the future.

The hotfix EEB for each version is available to download from the Downloads Center

 





 

 

 

== Prior to running these steps ==

Verify that the customer has not performed the log4j mitigation steps, and that the .war files are present:

Windows:
cd [install path]NetBackup\wmc\webserver
dir /a /S /b | findstr /r .war$

You should see at least the first 4 of these files at 8.1.2+  (ROOT.war may not exist)

C:\Program Files\Veritas\NetBackup\wmc\webserver\webapps\nbwebservice.war
C:\Program Files\Veritas\NetBackup\wmc\webserver\webapps_api\nbwss.war
C:\Program Files\Veritas\NetBackup\wmc\webserver\webapps_api\netbackup.war
C:\Program Files\Veritas\NetBackup\wmc\webserver\webapps_api\webui.war
C:\Program Files\Veritas\NetBackup\wmc\webserver\webapps_api_cssc\ROOT.war


Linux/Unix:
find /usr/openv/wmc/webserver -name "*.war"

You should see at least the first 4 of these files at 8.1.2+  (ROOT.war may not exist)


/usr/openv/wmc/webserver/webapps_api/netbackup.war
/usr/openv/wmc/webserver/webapps_api/nbwss.war
/usr/openv/wmc/webserver/webapps_api/webui.war
/usr/openv/wmc/webserver/webapps/nbwebservice.war
/usr/openv/wmc/webserver/webapps_api_cssc/ROOT.war


If these .war files are missing, you must either put them back from a backup or from a system that has not had the log4j mitigation steps performed, or apply the final fix log4j EEB before proceeding below:
https://www.veritas.com/support/en_US/article.100052058.html

== Non-Cluster Aware ==

  UNIX/Linux:

   1) /usr/openv/netbackup/bin/nbwmc -terminate
   2) /usr/openv/netbackup/bin/admincmd/nbcertconfig -u -i
   3) /usr/openv/netbackup/bin/admincmd/nbcertconfig -m
   4) On 8.0 and 8.1: /usr/openv/netbackup/bin/admincmd/nbcertconfig -t      
      On 8.1.1 and above:  /usr/openv/netbackup/bin/admincmd/nbcertconfig -t -f

   5) On 8.3 and above also run: /usr/openv/netbackup/bin/admincmd/nbcertconfig -s -f

   6) /usr/openv/wmc/bin/install/configureWmc
   7) /usr/openv/wmc/bin/install/configureCerts
   8) /usr/openv/wmc/bin/install/setupWmc
   9) /usr/openv/netbackup/bin/nbwmc -start
  10) /usr/openv/netbackup/bin/nbcertcmd -getCACertificate
  11) /usr/openv/netbackup/bin/nbcertcmd -getCertificate -force
      If the operation fails, perform the steps at "Create a token" section then return to this step.
  11) Remove the /usr/openv/var/global/vxss/nbcertservice/install_token file

  Windows:

   0) set WEBSVC_PASSWORD=<nbwebsvc password>
   1) C:\Windows\System32\sc.exe stop "NetBackup Web Management Console"
   2) <Install_Path>\NetBackup\bin\admincmd\nbcertconfig -u -i
   3) <Install_Path>\NetBackup\bin\admincmd\nbcertconfig -m
   4) On 8.0 and 8.1: <Install_Path>\NetBackup\bin\admincmd\nbcertconfig -t      
      On 8.1.1 and above: <Install_Path>\NetBackup\bin\admincmd\nbcertconfig -t -f

   5) On 8.3 and above: <Install_Path>\NetBackup\bin\admincmd\nbcertconfig -s -f

   6) <Install_Path>\NetBackup\wmc\bin\install\configureWmc
   7) <Install_Path>\NetBackup\wmc\bin\install\configureCerts
   8) <Install_Path>\NetBackup\wmc\bin\install\setupWmc
   9) C:\Windows\System32\sc.exe start "NetBackup Web Management Console"
  10) <Install_Path>\NetBackup\bin\nbcertcmd -getCACertificate
  11) <Install_Path>\NetBackup\bin\nbcertcmd -getCertificate -force
      If the operation fails, perform the steps at "Create a token" section then return to this step.
  11) Remove the <install_path>\NetBackup\var\global\vxss\nbcertservice\install_token file

== Cluster Aware ==

  • If system is UNIX/Linux, freeze the cluster beforehand.

  UNIX/Linux: Clustered Master Server: Active Node:

   1) /usr/openv/netbackup/bin/nbwmc -terminate
   2) /usr/openv/netbackup/bin/admincmd/nbcertconfig -u -i
   3) /usr/openv/netbackup/bin/admincmd/nbcertconfig -m
   4) On 8.0 and 8.1: /usr/openv/netbackup/bin/admincmd/nbcertconfig -t      
      On 8.1.1 and above:  /usr/openv/netbackup/bin/admincmd/nbcertconfig -t -f

   5) On 8.3 and above: /usr/openv/netbackup/bin/admincmd/nbcertconfig -s -f   
   6) /usr/openv/wmc/bin/install/configureWmc
   7) /usr/openv/wmc/bin/install/configureCerts
   8) /usr/openv/wmc/bin/install/setupWmc
   9) /usr/openv/netbackup/bin/nbwmc -start
  10) /usr/openv/netbackup/bin/nbcertcmd -getCACertificate
  11) /usr/openv/netbackup/bin/nbcertcmd -getCACertificate -cluster
  12) /usr/openv/netbackup/bin/nbcertcmd -getCertificate -cluster -force
  13) /usr/openv/netbackup/bin/nbcertcmd -getCertificate -force
      If the operation fails, perform the steps at "Create a token" section on this node then return to this step.
  14) Remove the /usr/openv/var/global/vxss/nbcertservice/install_token file

  UNIX/Linux: Clustered Master Server: Inactive Node:

   1) /usr/openv/netbackup/bin/nbcertcmd -getCACertificate
   2) /usr/openv/netbackup/bin/nbcertcmd -getCertificate -force
      If the operation fails, perform the steps at "Create a token" section on this node then return to this step.
   3) Remove the /usr/openv/var/global/vxss/nbcertservice/install_token file


  Windows: Clustered Master Server: Active Node:

   0) set WEBSVC_PASSWORD=<nbwebsvc password>
   1) Use Failover Cluster Manager to stop the "NetBackup Web Management Console"
   2) <install_path>\NetBackup\bin\admincmd\nbcertconfig -u -i
   3) <install_path>\NetBackup\bin\admincmd\nbcertconfig -m
   4) On 8.0 and 8.1: <Install_Path>\NetBackup\bin\admincmd\nbcertconfig -t      
      On 8.1.1 and above: <Install_Path>\NetBackup\bin\admincmd\nbcertconfig -t -f

   5) On 8.3 and above: <Install_Path>\NetBackup\bin\admincmd\nbcertconfig -s -f     
   6) <install_path>\NetBackup\wmc\bin\install\configureWmc
   7) <install_path>\NetBackup\wmc\bin\install\configureCerts
   8) <install_path>\NetBackup\wmc\bin\install\setupWmc
   9) Use Failover Cluster Manager to start the "NetBackup Web Management Console"
  10) <install_path>\NetBackup\bin\nbcertcmd -getCACertificate
  11) <install_path>\NetBackup\bin\nbcertcmd -getCACertificate -cluster
  12) <install_path>\NetBackup\bin\nbcertcmd -getcertificate -cluster -force
  13) <install_path>\NetBackup\bin\nbcertcmd -getcertificate -force
       If the operation fails, perform the steps at "Create a token" section on this node then return to this step.
  14) Remove the <install_path>\NetBackup\var\global\vxss\nbcertservice\install_token file

  Windows: Clustered Master Server: Inactive Node:

   1) <install_path>\NetBackup\bin\nbcertcmd -getCACertificate
   2) <install_path>\NetBackup\bin\nbcertcmd -getcertificate -force
       If the operation fails, perform the steps at "Create a token" section on this node then return to this step.
   3) Remove the <install_path>\NetBackup\var\global\vxss\nbcertservice\install_token file

Create a token:

Perform the following steps on the Master Server in order to get "nbcertcmd -getcertificate -force" finished successfully.

  a) For Cluster Aware and Non-Cluster Aware:

    UNIX/Linux:  /usr/openv/netbackup/bin/bpnbat -login -loginType WEB
    Windows:     <install_path>\NetBackup\bin\bpnbat -login -loginType WEB

    You will be prompted to enter the information as the following example.

  e.g.

    Authentication Broker [MasterServer1 is default]:
    Authentication port [0 is default]:
    Authentication type (NIS, NISPLUS, WINDOWS, vx, unixpwd, ldap) [unixpwd is default]:
    Domain [MasterServer1 is default]:  example.netbackup.com
    Login Name [root is default]:
    Password:

  b) For Cluster Aware and Non-Cluster Aware:

    UNIX/Linux:  /usr/openv/netbackup/bin/nbcertcmd -createToken -name <token_name> -reissue -host <Master server name>
    Windows:     <Install_Path>\netbackup\bin\nbcertcmd -createToken -name <token_name> -reissue -host <Master server name>

 e.g. nbcertcmd -createtoken -name token1 -reissue -host MasterServer1

    Token EFITVNDRKTWHXRCM created successfully.

  c) For Non-Cluster Aware:

      UNIX/Linux: /usr/openv/netbackup/bin/nbcertcmd -getCACertificate
                  /usr/openv/netbackup/bin/nbcertcmd -getCertificate -token <token_ID> -force

      Windows:    <install_path>\NetBackup\bin\nbcertcmd -getCACertificate
                  <install_path>\NetBackup\bin\nbcertcmd -getCertificate -token <token_ID> -force

    e.g. nbcertcmd -getcertificate -token EFITVNDRKTWHXRCM -force

For Cluster Aware:

      nbcertcmd -getCACertificate
      nbcertcmd -getCACertificate -cluster
      nbcertcmd -getCertificate -cluster -token <token_ID> -force
      nbcertcmd -getCertificate -token <token_ID> -force

   

Workaround on Media Servers and Clients:

   Perform the following commands on each Media Server and Client to obtain the new certificate. 

   UNIX/Linux: /usr/openv/netbackup/bin/nbcertcmd -getCACertificate
                  /usr/openv/netbackup/bin/nbcertcmd -getCertificate -force

      Windows:    <install_path>\NetBackup\bin\nbcertcmd -getCACertificate
                  <install_path>\NetBackup\bin\nbcertcmd -getCertificate -force

Notes: In case NetBackup Clients are in cluster environments, perform the above commands on each node.

 

Veritas Technologies LLC is aware that the above-mentioned issue (Etrack 3966961) is present in the current version(s) of the product(s) mentioned in this article. Veritas is committed to product quality and satisfied customers.

Please note that Veritas reserves the right to remove any fix from the targeted release if it does not pass quality assurance tests or introduces new risks to overall code stability. Veritas' plans are subject to change and any action taken by you based on the above information or your reliance upon the above information is made at your own risk.

Was this content helpful?