MS-SQL client sends job data to incorrect master

Problem

After switching an MS-SQL client to be backed up by a new Master Server, backups start and complete correctly on the new Master.

However, the MS-SQL client may mistakenly send some Job Progress updates to the old Master Server.

These updates appear in Activity Monitor on the old Master Server as a failed job.

The Detailed Status appears similar to:
Mar 17, 2022 11:01:35 PM - Error bprd (pid=17144) Unable to write progress log <path to progress log> on client <clientName> ...
Mar 17, 2022 11:01:35 PM - Error bprd (pid=17144) CLIENT <clientName> POLICY <policyName> SCHED NONE EXIT STATUS 46 (server not allowed access)

No other Job Details are filled in.
 

Error Message

The start of the client-side dbclient log will show that it is connecting to the proper new Master Server:
10:55:11.563 [16032.26648] <4> VxBSAInit: Veritas NetBackup XBSA Interface - 8.3.0.1  0 Log Init Info 2
10:55:11.563 [16032.26648] <4> VxBSAGetEnv: INF - entering GetEnv - NBBSA_API
10:55:11.563 [16032.26648] <4> VxBSAGetEnv: INF - returning -
10:55:11.563 [16032.26648] <4> VxBSAGetEnv: INF - entering GetEnv - NBBSA_FEATURE_ID
10:55:11.563 [16032.26648] <4> VxBSAGetEnv: INF - returning - <FEATURE_ID>
10:55:11.563 [16032.26648] <4> VxBSAGetEnv: INF - entering GetEnv - BSA_SERVICE_HOST
10:55:11.563 [16032.26648] <4> VxBSAGetEnv: INF - returning - newMaster            <- Here
10:55:11.563 [16032.26648] <4> bsa_checkFeatureId: INF - entering bsa_checkFeatureId
10:55:11.563 [16032.26648] <4> VxBSAGetEnv: INF - entering GetEnv - BSA_SERVICE_HOST
10:55:11.563 [16032.26648] <4> VxBSAGetEnv: INF - returning - newMaster            <- Here

During the job function 'CAgentMetadata' sends Job Progress data to the Master Server.
In this case, 'CAgentMetadata' sometimes sends data to the proper Master, and sometimes it sends data to the old Master.

11:01:30.579 [16032.26648] <4> CAgentMetadata::createMetadataImagesForClientInstance: INF - Creating 14 metadata objects for instance MSSQLSERVER and client clientName
11:01:30.595 [16032.26648] <2> do_pbx_service: [vnet_connect.c:3718] via PBX bprd CONNECT FROM <client_IP_Address>.51458 TO <newMaster_IP_Address>.1556 fd = 1448
11:01:30.735 [16032.26648] <2> CAgentMetadata::createMetadataImagesForClientInstance: DBG - Adding metadata to image for [clientName\MSSQLSERVER] [VNBU0-2222-3333-4444444444]
11:01:30.751 [16032.26648] <2> do_pbx_service: [vnet_connect.c:3718] via PBX bprd CONNECT FROM <client_IP_Address>.51461 TO <oldMaster_IP_Address>.1556 fd = 1376
 

Cause

An MS-SQL backup job stores job related metadata for the duration of active jobs in a mini client-side database.
The mini database files may contain leftover metadata that isn't completely flushed:
    <installPath>\Veritas\NetBackup\DbExt\MsSql\BackupHistory.d.db
    <installPath>\Veritas\NetBackup\DbExt\MsSql\BackupHistory.a.db

As long as there are no active jobs on the client, these mini database files can be safely deleted.
They will be rebuilt at the start of the next MS-SQL backup job.

Solution

A reboot of the problem client has been known to resolve this issue.

If a reboot is not possible, follow these instructions:

1. In Activity Monitor on the new Master, verify there are no active backup jobs running for the problem client.  If there are active jobs, either allow the jobs to complete, or kill the active jobs.

2. On the problem client, stop all NetBackup services.

3. On the problem client, delete the following 4 files: 
 - <installPath>\Veritas\NetBackup\DbExt\MsSql\BackupHistory.a.db
 - <installPath>\Veritas\NetBackup\DbExt\MsSql\BackupHistory.a.db.lck
 - <installPath>\Veritas\NetBackup\DbExt\MsSql\BackupHistory.d.db
 - <installPath>\Veritas\NetBackup\DbExt\MsSql\BackupHistory.d.db.lck

4. On the problem client, start all NetBackup services

5. From the new Master, run a MS-SQL backup of the client, which previously directed Job Details to the incorrect Master - observe if the behavior is corrected.