Event 41533 - EVDatabaseUpgrader Failed - "Cannot insert the value NULL into column 'MbxIdentity', table 'EnterpriseVaultDirectory.dbo.PstMigrationHistory'; column does not allow nulls"

Problem

While upgrading to EV 11.0.1 the EVDatabaseUpgrader fails while processing the directory database.  Error states "Failed to upgrade Directory database" and the DB_Upgrade_Log.txt specifies an exception "The EV11.0.1 EVDatabaseUpgrader.exe fails to complete, while updating the PSTMigrationHistory table in the Directory Database."

Error Message

---------Start Event Text---------
Type: Error
Event: 41533
Source: Enterprise Vault
Category: Database Upgrader
Description:
EVDatabaseUpgrader failed.

Failed to upgrade Directory database. Database:[EnterpriseVaultDirectory] Server:[SQLServer].

Error in the "EVDatabaseUpgrader YYYY-MM-DD HH-MM-SS.log" log:
1429 14:12:37.954 <2788> {DIRECTORYDB.EN_US} EXCEPTION: Failed to upgrade Directory database. Database:[EnterpriseVaultDirectory] Server:[SQLServer].
Symantec.EnterpriseVault.EVDatabaseUpgrader.ExecuteCommandException: Failed to execute command. Script: [D:\Program Files (x86)\Enterprise Vault\DirectoryDB_11_SchemaUpdates.sql]
Line: [934] Command: [IF(NOT EXISTS (SELECT * FROM syscolumns WHERE name='MbxIdentity' AND id = OBJECT_ID('PstMigrationHistory')))
BEGIN
EXEC Util_AddColumn @TableName='PstMigrationHistory', @ColumnName='MbxIdentity', @Type='int', @Nullable=1

EXEC('
UPDATE
PH
SET
PH.MbxIdentity = EME.MbxIdentity
FROM
ExchangeMailboxEntry EME
INNER JOIN
dbo.PstMigrationHistory PH
ON
PH.ExchangeMailboxEntryId = EME.ExchangeMailboxEntryId
')
EXEC Util_AlterColumn @TableName='PstMigrationHistory', @ColumnName='MbxIdentity', @DataType='int', @Nullable=0

-- Drop the Entry ID column as we have corresponding Indentity column
EXEC Util_DropColumn @TableName='PstMigrationHistory', @ColumnName='ExchangeMailboxEntryId'
END]. ---> System.Data.SqlClient.SqlException: Cannot insert the value NULL into column 'MbxIdentity', table 'EnterpriseVaultDirectory.dbo.PstMigrationHistory'; column does not
allow nulls. UPDATE fails. 

---------End Event Text---------

Cause

The issue occurs when there are records in the "PSTMigrationHistory" table without a matching ExchangeMailboxEntry Record.
 

Issue Confirmation

1. Login to the SQL server containing the EnterpriseVaultDirectory database
2. Open SQL server management studio
3. Open a new query window
4. Execute the following SQL query agaist the EnterpriseVaultDirectory database
   
   USE EnterpriseVaultDirectory
Select Count(*) 'PST Files(Qty)'
FROM PSTMigrationHistory
WHERE ExchangeMailboxEntryId NOT IN (SELECT ExchangeMailboxEntryId FROM 
ExchangeMailboxEntry)
    
5. If the quantity returned is greater than zero, the issue is present in the environment and a case with support will need to be opened

If the issue was encountered during an upgrade confirm the last error is as follows

1. Login to the Enterprise Vault Server where the EVDatabaseUpgrader was run
2. Using windows explorer navigate to the location "<EnterpriseVault_Install_Path>\Reports"
3. Locate and open the most recent EVDatabaseUpgrader log file "EVDatabaseUpgrader YYYY-MM-DD HH-MM-SS.log"
4. Search for "EXCEPTION:"
5. Confirm the exception states the following:
   
   {DIRECTORYDB.EN_US} EXCEPTION: Failed to upgrade Directory database. Database:[EnterpriseVaultDirectory] Server:[SQLServer]. 
   Symantec.EnterpriseVault.EVDatabaseUpgrader.ExecuteCommandException: Failed to execute command. Script: [<EV_Install_Path>\DirectoryDB_11_SchemaUpdates.sql]
    
6. Confirm that further down in the error that the exact following text is present
   
   Cannot insert the value NULL into column 'MbxIdentity', table 'EnterpriseVaultDirectory.dbo.PstMigrationHistory'; column does not allow nulls. UPDATE fails.

Solution

1. Confirm the issue is present using the steps provided above
2. Locate and restore a backup of the EnterpriseVaultDirectory Database directly before the upgrade
3. Download Enterprise Vault 11.0.1 CHF 2, but DO NOT install the hotfix
4. Extract the "Veritas_Enterprise_Vault_11_0_1_CHF2.iso" file
5. Open the ISO file by either mounting the ISO or opening with a program similar to WinZip or 7Zip
6. Navigate to the "Veritas Enterprise Vault\Server\Hotfixed Files\%VAULT%" folder
7. Copy the DirectoryDB_11_SchemaUpdates.sql file to the local machine
8. Login to the server where the DirectoryDB_11_SchemaUpdates.sql will be run
9. Using Windows Explorer navigate to the Enterprise Vault installation directory
10. Rename the DirectoryDB_11_SchemaUpdates.sql file to DirectoryDB_11_SchemaUpdates.sql.old
11. Copy the DirectoryDB_11_SchemaUpdates.sql file into the Enterprise Vault Installation Directory
12. Run the EVDatabaseUpgrader.exe