NetBackup Java GUI fails to login, status 3657

Problem

The NetBackup Java GUI login failure can occur after trying to change permissions, the service user or after upgrades.  This specific article is related to the Java GUI not logging in after a NetBackup Flex instance upgrade of the primary server from NetBackup 10.2.0.1 to 10.4 experienced difficulties.   There were issues and complications that resulted in actions being performed, to enable the NetBackup system to work correctly after the upgrade.   On this occasion it's not clear exactly what changed the permissions, resulting in the Java GUI not enabling the login.

Note:  The same error 3657, will occur if nbwmc is not running.  Please confirm nbwmc is running before continuing to troubleshoot.

The Web UI will be available and can be used as an alternative to administer NetBackup.

Error Message

Java GUI login error pop-up window:

Unable to login, status: 3657
Failed to check whether multifactor
authentication is enabled for the user
account or not.

bpjava-msvc log:

17:14:45.632 [143992] <2> connect_to_credcache_mgr: No active Credential Cache Manager found to connect to.
17:14:45.632 [143992] <16> vnet_credwrite: Failed to establish connection with Credential Cache Manager.
17:14:45.632 [143992] <16> writeSessionToken_i: (unix)CredWrite[webtoken.dat] return value = 14
17:14:45.632 [143992] <16> LoginWithCertManager::saveNewToken: WriteSessionToken for tokenId [webtoken.dat] failed with status = 14
17:14:45.632 [143992] <16> LoginWithCertManager::saveNewToken: Exception encountered, status code: 14
17:14:45.632 [143992] <16> LoginWithCertManager::getToken: save new token failed with error: 14
17:14:45.632 [143992] <16> LoginWithCertManager::getToken: Exception encountered, status code: 14
17:14:45.632 [143992] <16> NBClientCURL::performCurlOperation: Failed to fetch authorization token. Actual Error code 14return code :5930
17:14:45.632 [143992] <16> curlSendRequest: failed to perform curl request for URI :netbackup/security/mfa-token retval :5930
17:14:45.632 [143992] <16> nbcert_curl_mfa_get_token: Failed to send request for mfa-token, error: 5930
17:14:45.632 [143992] <2> NBClientCURL:~NBClientCURL: Performing curl_easy_cleanup()
17:14:45.632 [143992] <2> nbclnt_curl_prefnet::reset: Returning VN_STATUS_SUCCESS
17:14:45.632 [143992] <16> isUserMFARegistered: MFA state token generation failed: [5930]
17:14:45.632 [143992] <2> writetagEx: tag= BAD_STATUS, sockFd = 0, tagged output = nope
17:14:45.632 [143992] <2> writetagEx: tag character based, and limited to 4 chars
17:14:45.632 [143992] <2> writeCountOrDataStartEx: line count = 2 , sockFd = 0
17:14:45.632 [143992] <2> respond_with_STATUS: line 1 = 3657
17:14:45.645 [143992] <2> respond_with_STATUS: line 2 = Failed to check whether multifactor authentication is enabled for the user account or not.

Cause

Due to incorrect permissions on a critical directory, when NetBackup is being started there is an inability to create a socket file (cred_cache0), needed for the Java GUI authentication.

Solution

This issue is resolved in NetBackup 10.5 and later. For releases prior to NetBackup 10.5:

Check to determine if the socket file  /usr/openv/var/credcache/0/cred_cache0  exists or not.   If this file does not exist, then check the directory  /usr/openv/var/credcache/0  on the primary server to ensure the directory permissions only have read, write and execute for the owner root.

# ls -ld /usr/openv/var/credcache/0
drwx------. 2 root root 49 Jul  2 04:27 /usr/openv/var/credcache/0

The above example is valid, however, if the permissions or ownership are any other than that seen above, they will need to be changed.   To change the permissions and / or ownership use;

# chmod 700 /usr/openv/var/credcache/0
# chown root /usr/openv/var/credcache/0

Note:  Manually creating the socket file is not necessary, the permission inconsistencies need to be resolved and NetBackup restarted to solve the issue.
 

After resolving any permission / ownership problems, to recreate the missing socket file (cred_cache0), restart NetBackup on the primary server.

Note:  Another check will be to ensure the ' 127.0.0.1    localhost ' entry is defined correctly in the /etc/hosts file and this file does not contain unwanted or control characters which can affect the hostname resolution.   NetBackup needs localhost to resolve correctly to ensure successful communications to services.This does not mean to put the actual hostname on that line, merely to ensure that the name localhost is going to resolve for the localhost IP of 127.0.0.1.