NDMP backups fail after 8 hours with 'no mover progress' Status 99

Problem

NDMP backups fail after 8 hours of Filer inactivity

Error Message

06/04/2019 10:12:11 - connecting
06/04/2019 10:12:11 - connected; connect time: 0:00:00
06/04/2019 10:12:14 - Info bptm (pid=291411) using 30 data buffers
06/04/2019 10:12:14 - Info bptm (pid=291411) using 262144 data buffer size
06/04/2019 10:12:14 - Info bptm (pid=291411) start backup
06/04/2019 10:12:18 - Info ndmpagent (pid=291398) NDMP Remote disk
06/04/2019 10:12:18 - Info ndmpagent (pid=291398) Filer101: Direct Access Restore information is supported
06/04/2019 10:12:19 - begin writing
  Note: There is an 8 hour time gap here.
06/04/2019 18:12:22 - Error ndmpagent (pid=291398) aborting operation - no mover progress
06/04/2019 18:12:22 - Error ndmpagent (pid=291398) NDMP backup failed, path = /Filer/Path/
06/04/2019 18:12:27 - Info bptm (pid=291411) EXITING with status 99

Cause

The default NetBackup behavior is an 8-hour timeout value when waiting for Filer NDMP operations to update the NetBackup server on its operations

Solution

Recommend: Contact the Filer support team to investigate the Filer progress hang.

Workaround: Increase the default NetBackup timeout temporarily to determine if the job will complete with more time. The steps below should help with accomplishing this task:

1. Modify the default NetBackup timeout value by creating the NDMP_PROGRESS_TIMEOUT file on the NetBackup media server, in the following path:
   • Unix: /usr/openv/netbackup/db/config/
   • Windows: install_path\Veritas\NetBackup\db\config
2. The file must only contain a single number, which is the desired timeout value, in minutes.
3. The timeout value can be modified by executing the following command:
   echo 1440 > NDMP_PROGRESS_TIMEOUT
   Note: This command will set the timeout to 24 hours (1440 minutes).
4. Then, stop and re-start NetBackup services.