VMware backups are failing with status 11

Problem

The backup job would complete writing data and the bpbkar logs reports an EXIT STATUS 0, however while trying to upload the Catalog metadata the Job fails with status 11.
 

Error Message

Job Details:

Jan 29, 2021 10:52:25 AM - begin writing
Jan 29, 2021 10:53:06 AM - Info bpbkar32 (pid=4936) INF - Transport Type =  san
Jan 29, 2021 10:57:45 AM - Info bptm (pid=13176) waited for full buffer 4877 times, delayed 7607 times
Jan 29, 2021 10:57:45 AM - Error bpbrm (pid=11832) db_FLISTsend failed: system call failed (11)
Jan 29, 2021 10:57:53 AM - Info bpbkar32 (pid=0) bpbkar waited 15 times for empty buffer, delayed 28 times.
Jan 29, 2021 10:57:53 AM - Info bpbkar32 (pid=4936) done
Jan 29, 2021 10:57:53 AM - Info bpbkar32 (pid=0) done. status: 11: system call failed
Jan 29, 2021 10:57:53 AM - end writing; write time: 0:05:28
system call failed  (11)

Snippets from the bpbpkar logs.

10:57:45.192 [4936.9056] <2> tar_base::backup_finish: TAR - backup:                     205 files
10:57:45.192 [4936.9056] <2> tar_base::backup_finish: TAR - backup:          file data:          0 bytes
10:57:45.192 [4936.9056] <2> tar_base::backup_finish: TAR - backup:         image data:  941667328 bytes  47 gigabytes
10:57:45.192 [4936.9056] <2> tar_base::backup_finish: TAR - backup:       elapsed time:        323 secs    159156449 bps
10:57:45.192 [4936.9056] <2> tar_base::V_vTarMsgW: INF - Total Size:0
10:57:45.192 [4936.9056] <2> dtcp_write: TCP - success: send socket (612), 19 of 19 bytes
10:57:45.255 [4936.9056] <2> tar_base::V_vTarMsgW: INF - EXIT STATUS 0: the requested operation was successfully completed
 

Snippets from the bpbrm logs.

10:57:45.270 [11832.12980] <2> closeConnection: Caching connection for query type 78 for reuse
10:57:45.270 [11832.12980] <2> db_FLISTsend: db_end_sts() failed: system call failed (11)
10:57:45.270 [11832.12980] <16> non_mpx_backup_archive_verify_import: db_FLISTsend failed: system call failed (11)
10:57:45.270 [11832.12980] <2> set_job_details: Tfile (496425): LOG 1611898065 16 bpbrm 11832 db_FLISTsend failed: system call failed (11)
10:57:45.270 [11832.12980] <2> send_job_file: job ID 496425, ftype = 3 msg len = 75, msg = LOG 1611898065 16 bpbrm 11832 db_FLISTsend failed: system call failed (11)
10:57:45.286 [11832.12980] <4> db_error_add_to_file: db_FLISTsend failed: system call failed (11)
10:57:45.301 [11832.12980] <2> bpbrm kill_child_process_Ex: start
10:57:50.895 [11832.12980] <2> bpbrm wait_for_child: start
10:57:50.895 [11832.12980] <2> bpbrm wait_for_child: child exit status: 82
10:57:50.895 [11832.12980] <2> bpbrm wait_for_child: child not active
10:57:50.895 [11832.12980] <2> inform_client_of_status: INF - Server status = 11


Tracing the above connection from bpbrm to bpdbm in the bpdbm logs

10:57:45.223 [6948.5224] <16> CatImg_fopen_format: CatImg_OpenCatFile failed for file: E:\Program Files\NetBackup\NB_IMAGEDB\<vm_name>\\1611000000\tmp\<vm_name>_1611897740_FULL.f
10:57:45.223 [6948.5224] <16> Function: NBE_CatFile_Bin::InitMMFiles(): unable to create catstore directory \\?\E:\Program Files\NetBackup\NB_IMAGEDB: Error:  File: ../nbe_cat_file_bin.cpp Line: 1250  Error Number: 1
10:57:45.223 [6948.5224] <16> add_files_alt: Error opening catalog image file 11

Cause

The contents of the ALTPATH file had an extra trailing '\'. 

ALTPATH
E:\Program Files\NetBackup\NB_IMAGEDB\<vm_name>\

Solution

Removed the '\' in the end and run a backup.

Example:

E:\Program Files\NetBackup\NB_IMAGEDB\<vm_name>\

TO
E:\Program Files\NetBackup\NB_IMAGEDB\<vm_name>