How to set up building blocks and configure Enterprise Vault for a failover scenario in Enterprise Vault 14.2 or later

Description

The following procedure should be performed on a new Enterprise Vault installation (Multiple Enterprise Vault servers within the same Enterprise Vault site).

1. Ensure you have DNS aliases for each Enterprise Vault Server that point to the physical server's DNS A record.
   
   Example

   DNS Alias Configuration
   PNEV.RGPCert.Com  CNAME   Pune.RGPCert.Com
   SCEV.RGPCert.Com   CNAME   SantaClara.RGPCert.Com
   NYEV.RGPCert.Com  CNAME   NewYork.RGPCert.Com

   Physical Servers (Computer Hostname)
   Pune.RGPCert.Com       A HOST 192.168.1.3
   SantaClara.RGPCert.Com    A HOST 192.168.1.4
   NewYork.RGPCert.Com  A HOST 192.168.1.5

2. Ensure you have an EVSite DNS alias pointing to a designated primary Enterprise Vault server.
   
   Example
   EVSite.RGPCert.Com CNAME  PNEV.RGPCert.Com
    
3. We recommend to use a DNS alias for the SQL database server as well (This will require in the DR scenario).
   
   Example

   DNS Alias Configuration
   EVSQL.RGPCert.Com  CNAME   RGPSQL.RGPCert.Com

   Physical Servers (Computer Hostname)
   RGPSQL.RGPCert.Com       A HOST 192.168.1.2
    

4. Check that these records resolve correctly on each Enterprise Vault Server's, and flush the DNS cache if required (ipconfig /flushdns).
5. The self-signed certificate created by the Enterprise Vault configuration does not include the names of the other Enterprise Vault servers in the building blocks configuration. You need to create a replacement certificate to include subject alternative names / DNS alias of all EV servers in a certificate.
   For more details – Refer - Enterprise Vault™ Requesting and Applying an SSL Certificate Default Web Site in IIS on all EV servers, add/replace the certificate created above.
6. Perform the Pre-installation Tasks and install Enterprise Vault as described in the Enterprise Vault documentation. Remember to use the alias name and not the machine name when configuring Enterprise Vault. (This is mandatory for building blocks to work).
7. While configuring Enterprise Vault, make sure all data locations should be accessible from all EV servers and if EV servers are located at different physical locations, then data should be replicated.
   These data locations must be accessible between the associated building blocks servers:
   
   Example:
   In the below scenario \\RNAS is accessible from both locations.
   In the case of DR site – \\RNAS should be replicated to other locations with the maintaining same DNS name. And SQL database also needs to replicate or use SQL AlwaysOn.
   
   Here are Enterprise Vault common locations are:
   
   Storage queue location - The Enterprise Vault Storage queue location must be on a file system that is a common shared location between the associated building block servers. (Figure-1)
   Figure 1:
   
   
   Shopping Location - Enterprise Vault server is running in a building blocks environment and is hosting another Enterprise Vault server. The Shopping Service data file locations must match the same location: (Figure-2)
   Figure 2:
   
    

   Indexing and Metadata data locations must be in a location that appears as a local folder to the computer; for example, iSCSI or SAN locations are acceptable.

   Important Note - Before running USL for Building Block make sure to replicate these locations and make it available to other EV servers participating in building blocks. We recommend block-level replication such as InfoScale VVR or Storage Replica etc to avoid inconsistency in data at both locations. 

   In the below scenario Drive R: is local to the Pune EV server and Drive S: is local to the SantaClara EV server. Block level replication (check direction) is configured for building blocks. (Figure-3) (Same should be followed for DR scenario).
   Figure 3
   
   
   Index Location - Elasticsearch Index Location - Starting with release 14.2, Enterprise Vault adds Elasticsearch as the new indexing engine which supports only one index data location. (Figure-4)
   Figure 4
   

   Index metadata location - The location of indexing configuration and reporting data. (Figure-5)
   Figure-5
   
   Snapshots of Elasticsearch indexes – You can use the \\RNAS shared as snapshot location.
   Refer - https://www.veritas.com/support/en_US/article.100051446 

   Note – Enterprise Vault EV 14.1 and earlier can have index location on the shared path (example \\RNAS\indexes )
   
   Cache Location - Each Enterprise Vault server that hosts an Indexing service requires a server cache. You must specify the server cache location on the Cache tab of the computer properties for all the Enterprise Vault servers. (Figure-6)
   Figure-6
   

   Vault Store Partitions – Vault partitions should not be created locally on the EV server using NTFS for the EV servers participating building blocks.

   For important information about other requirements, see the section "Extra requirements for building blocks" in the Enterprise Vault Administrator's Guide.

8. Once all Enterprise Vault servers have successfully installed, complete the Post installation Tasks as described in the above steps and Enterprise Vault documentation.
9. Notice that the VAC shows both the computer alias and machine name for each computer. (Figure-7)
   Figure-7
   
   
   Fail-Over
   The following steps need to be taken when an EV server has failed or been replaced, and the services need to be moved:
    
10. Before initiating USL, ensure that:
    • Only the Enterprise Vault Admin and Enterprise Vault Directory services are running on the EV servers.
      The Admin service needs to be running so that USL can determine the machine name and hence detect if the alias has changed. It also needs this service to check whether a single machine is represented by multiple computer aliases.
    • Check the storage queue pending and total length and wait till it becomes zero.
    • Stop all other Enterprise Vault services (i.e. Storage, Index, Task Controller, and Shopping Services) on both EV servers (PNEV & SCEV).
    • Stop and Disable the FSRM (File Server Resource Manager) service if running.
    • Switch the replication direction from the First (Pune) Enterprise Vault Server to another (SantaClara) Enterprise Vault Server and wait till it replicates complete data and is available.
11. Run Update Service Locations (USL), follow these steps:
    • Change the DNS alias of the Failed Enterprise Vault server pointing to running Enterprise Vault Server.
      Example:
      PNEV.RGPCert.Com  CNAME   Pune.RGPCert.Com
      To
      PNEV.RGPCert.Com   CNAME   SantaClara.RGPCert.Com
    • Start the Administration Console on any of the servers in the Enterprise Vault site.
    • In the left pane, expand the tree until the Enterprise Vault Servers container is visible.
    • Right-click the Enterprise Vault Servers container and, on the shortcut menu, click Update Service Locations.
      You are prompted for the password for the Vault Service account.
    • Enter the password for the Vault Service account and click OK.
      Enterprise Vault updates the service locations and creates new services as necessary.
12. Once USL has been completed, a summary page will be displayed.
    At the end of the update, a summary screen appears, listing the services on each computer on the site. (Figure-8)
    Figure -8
    
     
13. Refresh the 'Computers' node after each USL run so that it updates the changed information in the EV Administration Console. (Figure-9)
    Figure-9
    
     
14. Start Enterprise Vault services (i.e. Storage, Index, Task Controller, and Shopping Services) on running EV servers (SCEV).
15. To Verify failed server - click in the Administration Console click on First (PNEV) Enterprise Vault Server and verify services are showing hosted. (Figure-10)
    Figure-10
    
     
16. To Verify the running server - click in the Administration Console click on Running (SCEV) Enterprise Vault Server and verify services are showing running. (Figure-11)
    Figure-11
    
     

17. Check that you have DNS connections to the Directory database and all the Vault Stores. 
    The Event log will throw errors if it has any problem connecting to these databases.
     
    You are done with Building Block.
    Now you can recall the items from failed Enterprise Vault server via running Enterprise Vault Server as they both are in Building Block.

    Note – When failed EV server comes back online and would like to Fail-Back then follow the reverse process mentioned in steps 10 to 17 in this article.

Related Knowledge Base Articles

• How to setup building blocks and configure Enterprise Vault for a failover scenario in Enterprise Vault 14.1 or earlier
• How to configure Enterprise Vault building blocks (Update Service Locations) to implement a disaster recovery solution using Microsoft Cluster
• Enterprise Vault™ 14.2 Administrator's Guide - Failover in a building blocks configuration