After upgrading Exchange 2013 to Cumulative Update 11 (CU11), Exchange backup may fail with either status code 69 or 130

Problem

After upgrading Exchange 2013 to Cumulative Update 11 (CU11), Exchange backup may fail with either status code 69 or 130.
 

Error Message

Scenario #1: Backup job fails with status code 69
bpbkar log:
12:24:03.806 [64224.31176] <2> tar_base::V_vTarMsgW: ERR - Streaming backups are invalid with Exchange 2010 or 2013.
12:24:03.806 [64224.31176] <2> dtcp_write: TCP - success: send socket (520), 64 of 64 bytes
12:24:03.806 [64224.31176] <4> dos_backup::V_VerifyFileList: INF - UBS Local Type for 'Microsoft Exchange Database Availability Groups:\' --> 10020044
12:24:03.806 [64224.31176] <4> dos_backup::V_VerifyFileList: INF - UBS Local Type for 'Microsoft Exchange Database Availability Groups:\dagname' --> 002a0000
12:24:03.806 [64224.31176] <4> dos_backup::V_VerifyFileList: INF - UBS Local Type for 'Microsoft Information Store:\' --> 1002002a
12:24:03.806 [64224.31176] <4> dos_backup::V_VerifyFileList: INF - UBS Local Type for 'Microsoft Information Store:\db1\' --> 10200001
12:24:03.806 [64224.31176] <2> tar_base::V_vTarMsgW: INF - EXIT STATUS 69: invalid filelist specification

Scenario #2: Snapshot job fails with status code 130
bpfis log:
15:22:55.758 [33200.2000] <2> onlfi_vfms_logf: INF - WRN - Check_Ex2013_BCD: Unable to find Exchange database <db2> in the Backup Components Document
15:22:55.758 [33200.2000] <2> onlfi_vfms_logf: INF - Status FS_WRITER_METADATA_FAILURE (0xE000FEC9) generating Writer Metadata Document stream in SHADOW::ReadVcb
15:22:55.758 [33200.2000] <2> onlfi_vfms_logf: INF - ERR - EnumerateForLocalMetaData: FS_ReadObj(1) Failed! (0xE000FEC9:A failure occurred accessing the Writer metadata.
15:22:55.773 [33200.2000] <16> bpfis main: FTL - snapshot creation failed - Error attempting to gather metadata., status 130

NOTE : To generate the logs, both bpbkar and bpfis folders must exist on the affected Exchange mailbox server under <install dir>\Veritas\NetBackup\logs\. If they do not exist already, create them and re-run the backup.

 

Cause

In Exchange versions prior to 2013 CU11, Exchange Management Shell session typically connects to the local server where the shell is opened, and to a proxy only when required.

Exchange 2013 CU11 introduces a new functionality called mailbox anchoring: Instead of connecting locally, Exchange Management Shell session will now be proxied to the server where the user’s mailbox is located. If the user does not have a mailbox, an arbritration mailbox is used instead and the session will be proxied to the server where that arbitration mailbox is located. This new behavior may cause backup to fail, since part of the process involves querying Exchange topology using PowerShell.

For more information on mailbox anchoring, refer to:
  http://blogs.technet.com/b/exchange/archive/2015/12/15/exchange-management-shell-and-mailbox-anchoring.aspx.

Note: This behavior has been reverted in Exchange 2013 CU12.  This issue does not affect Exchange 2016.

 

Solution

Workaround:
If applying 2013 CU12 or above is not an option, the following procedure may be performed on all Exchange mailbox servers:

1. Launch Registry Editor (regedit).
2. Browse to HKEY_LOCAL_MACHINE > SOFTWARE > Veritas > NetBackup > BEDS > Engine > Exchange.
3. On the right hand pane, create a new DWORD (32-bit) value. For the name, type:  PowerShellOptions, and hit Enter.
4. Double click PowerShellOptions and enter 1 as the value data. Hit Enter again to save.

Setting PowerShellOptions to 1 forces monad (the NetBackup process responsible for running PowerShell queries) to connect to local Exchange Management Shell session, thereby bypassing the need to interact with mailbox anchoring.

Note:  
If using Exchange 2013 CU11, do not set the NetBackup PowerShellOptions Registry value unless GRT backups fail to catalog mailboxes.
If GRT backups fail to catalog mailboxes with Exchange 2013 CU11 and you have archive mailboxes  please upgrade to CU12.