Arctera Data Insight Administrator's Guide

1. Data Insight Indexer
2. Data Insight Collector components
3. Data Insight Management Server
4. Removing oldData Insight

Migration Steps

1. Disable Device Scanning on the Data Insight Collector nodes.

   • To disable File System Full Scan

     • Navigate to Settings > Data Insight Servers > Collector node > Advanced settings.

     • Expand the Filesystem Scanner settings wizard and locate settings for Full Scans.

     • From the drop-down, set the value of Override default full scan schedule schedule to NEVER.

       

   • To disable File System Incremental Scan,

     • Navigate to Settings > Data Insight Servers > Collector node > Advanced settings.

     • Expand the Filesystem Scanner settings wizard and locate settings for Incremental Scans.

     • From the drop-down, set the value of Override default incremental scan schedule to NEVER.

       

   • c) To disable SharePoint Full Scan,

     • Navigate to Settings > Data Insight Servers > Collector node > Advanced settings.

     • Expand the SharePoint settings wizard.

     • From the drop-down, set the value of Override default full scan schedule schedule to NEVER.

       

   • To disable SharePointOnline Incremental Scan,

     • Navigate to Settings > Data Insight Servers > Collector node > Advanced settings.

     • Expand the SharePoint settings wizard.

     • From the drop-down, set the value of Override default SharePointOnline incremental scan schedule to NEVER.

       

     • Repeat same steps for all collector nodes.

2. Disable Device Auditing on the Data Insight Collector nodes.

   • Remote desktop to Collector node and to disable auditing of

     • NetApp Cluster File Server devices monitored by given collector, stop DataInsightFpolicyCMod service on collector node. You can do this by using Services.msc on windows.

     • NetApp 7 mode devices monitored by given collector, stop DataInsightFpolicy service on collector node. You can do this by using Services.msc on windows.

     • EMC Celerra, EMC Isilon, EMC Unity devices monitored by given collector, stop DataInsightCelerra service on collector node. You can do this by using Services.msc on windows.

     • Generic devices monitored by given collector, stop DataInsightGenericCollector service on collector node. You can do this by using Services.msc on windows.

     • OneDrive devices monitored by given collector, stop DataInsightOneDrive service on collector node. You can do this by using Services.msc on windows.

     • SharePointOnline devices monitored by given collector, stop DataInsightSharePointOnline service on collector node. You can do this by using Services.msc on windows.

     • AmazonS3 devices monitored by given collector, stop DataInsightAmazonS3 service on collector node. You can do this by using Services.msc on windows.

   • To disable auditing of HNAS devices monitored by given collector,

     • Navigate to Settings > Filers.

     • Select HNAS device associated with given collector and click device name.

     • Click Edit and locate Event Monitoring setting.

     • Uncheck Enable File System Event Monitoring box.

     • Repeat this process for all the HNAS devices.

       

     • To disable auditing of all Windows File Server and MSCS Cluster File Server,

       • Navigate to Settings > Filers.

       • Select Windows File Server associated with given collector and click device name.

       • Click Edit and locate Event Monitoring setting.

       • Uncheck Enable File System Event Monitoring box.

       • Repeat this process for all the Windows File Server devices.

         

     • To disable auditing of SharePoint webapps,

       • Navigate to Settings > Filers.

       • Click Collector node > Advanced settings.

       • Expand the SharePoint settings wizard.

       • From the dropdown, set the value of Override default schedule to fetch audit events from SharePoint to NEVER.

         
3. Classification

   Ensure that no Classification requests are in progress. If any classification request is in progress, do not cancel the request. Wait for the request to complete. As for the Pending requests, you can cancel all of them.

4. Self Service Portal

   Ensure that no Self Service Portal Workflow requests are in progress. If any Self-Service Portal Workflow request is in progress, do not cancel the request. Wait for the requests to complete.

5. Workflow

   Ensure that no Workflow requests are in progress. If any Workflow requests is in progress, do not cancel the request. Wait for the requests to complete.

6. Reports

   • Scheduled and Ad hoc reports: Ensure that none of the Data Insight Reports are running or scheduled to run on Indexer nodes or Management Server. If scheduled or ad hoc report generation is in progress, do not cancel it. Wait for the process to complete and then disable all reports.

   • Dashboard: Ensure that Data Insight Dashboard report execution is disabled or set to Never. If Dashboard report is in progress, do not cancel it. Wait for the process to complete and then disable all reports. To disable Dashboard Summary report,

     • Navigate to Settings > Data Insight Servers > Advanced Analytics > Configuration.

     • Click Edit > Summary Dashboard Configuration setting.

     • From the drop-down, set the Refresh schedule to NEVER.

       

   • User Risk

     • Ensure that UserRiskJob is disabled on all Indexer nodes and Management Server.

     • If UserRiskJob is in progress, do not cancel it. Wait for the process to complete and then disable the job.

     • •To disable the job,

       • oNavigate to Settings -> Data Insight Servers -> Management server node and click Jobs tab.

       • In the filter, search UserRiskJob

       • Click Select action and then click Disable.

       • Repeat this for all Indexer nodes.

     • Ensure that UserRiskMergeJob is disabled on the Management Server. If UserRiskMergeJob is in progress, do not cancel it. Wait for the process to complete and then disable the job. To disable the job,

       • Navigate to Settings > Data Insight Servers > Management server node and click Jobs tab.

       • Connect to Data Insight console, go to Settings > Data Insight Servers.

       • In the filter, search UserRiskMergeJob

       • Click Select action and then click Disable.

Migrating Data Insight Indexer

1. Identify total number of Indexers to be migrated to a new environment.
2. Deploy new Indexer nodes and register them with the Management Server.
3. To migrate individual Indexer,

   • Backlog on Collector: Identify the Collector node associated with this Indexer node and ensure that all the scan and audit files present in Outbox directory on the Collector node are transferred to Indexer node for processing. The default location of the Outbox directory on the Collector node is C:\DataInsight\data\outbox.

   • Ensure that there is no backlog on the Collector node.

   • Make sure that scanning and auditing for all the devices being monitored by given Indexer node is disabled.

   • If there are files to be processed in the C:\DataInsight\data\inbox directory, manually run IndexWriterJob on that Indexer by navigating to Data Insight console > Settings > Data Insight Servers > Indexer node > Jobs > IndexWriterJob > Select action > Run

   • After processing the backlog files, ensure that there is no active instance of IndexWriterJob or ActivityIndexJob. You can confirm this using Task Manager in Windows

   • Disable IndexWriterJob on the Indexer node. To disable IndexWriterJob, navigating to Data Insight console > Settings > Data Insight Servers > Indexer node > Jobs > IndexWriterJob > Select action > Disable.

   • Disable ActivityIndexJob on the Indexer node. To disable ActivityIndexJob, navigating to Data Insight console -> Settings -> Data Insight Servers -> Indexer node -> Jobs -> ActivityIndexJob -> Select action -> Disable.

4. To migrate Single indexer, identify the total number of devices being monitored by a given indexer node. To identify total number of devices,

   • Navigating to Data Insight console > Filers.

   • Go to By Indexer filter.

   • Check the box for the Indexer node.

   Once you apply the filter, list of devices being monitored by given indexer appears. Follow the steps below to migrate existing indexer to the new indexer:

   • In the Data Insight Management Console, click Settings > Filers.

   • Find the desired filer or device and click View under Select Action drop down.

     

   • On the Configuration page click Change Indexer.

     

   • On the Select Indexer page, click Select under the Action column on the new Indexer and click Yes on the Warning popup.

     

   • Click OK.

     

Check Indexer Migration Status

1. Now, Data Insight will start migrating filer to the new selected Indexer node. To check migration progress, click on . View Details
   
2. Migration process might take some time. Depending on the number of shares and the amount of data being transfer from the old Indexer to the new Indexer, migration process time may vary.
3. When files are being transferred from the Source Indexer to the Destination Indexer, the process might be considerably slower than expected. This is due to a pause schedule of MsuMigrationJob, which is set at 07:00 by default. If you want to expedite the process, edit the default schedule for the MsuMigrationJob from Daily at 07:00 to run every hour on the Source Indexer. To do that,

   • In the Data Insight Console, navigate to Settings > Data Insight Servers (source indexer) > Advanced Settings > Set Custom Properties

   • Add the following values in the given fields:

     Property Name =	job.MsuMigrationJob.cron
     Property Value =	0 0 0/1 * * ? *

   • You can confirm the settings in the Jobs list of the source indexer.

   • The job will now check the migration status each hour and restart the transfer if it stops.

   • After completing the migration, the New Indexer will appear on the same page.

     
4. Repeat steps for all the devices configured as part of the old indexer node.

Migrating all Devices to a new Collector

1. Identify total number of Data Insight Collectors to be migrated to a new environment. To identify total number of collectors,

   • Navigate to Settings > Data Insight Servers.

   • Go to Server type filter and apply Collectors and Indexer & Collector filter.

   • List of collectors will appear.

2. Deploy new Collector nodes by following the Data Insight Installation guide and register them with the Management Server. Make sure that the new Collector nodes and Management Server are running on the same version.
3. Migrate a Single Collector

   Pre-requisite:

   • Full and Incremental Scans are disabled on the Collector node. Please refer to #2 from the Prerequisite section.

   • Auditing is disabled for all devices being monitored by the Collector node. Please refer #3 from the Prerequisite section.

     To migrate every device from existing collector to new collector,

     • Navigate to Data Insight Management Console > Settings > Filers.

     • Find the desired filer or device and click Edit under Select Action drop down.

       

     • Click Change Collector.

       

     • Select new Data Insight Collector from the list and click Select.

       

     • On the Warning pop-up click Yes.

     • Once device is moved to new a Data Insight Collector, you can see the new assignment on the page.

       
4. Repeat Step #4 for all the devices configured as a part of old Data Insight Collector node. Move every device to a new Data Insight Collector.
5. If you have configured proxy settings for OneDrive, SharePoint Online or Amazon S3, then you need to copy proxy_server.properties stored on the old collector at following locations and paste it on the new collector.

   <installDIR>\connectors\amzons3, <installDIR>\connectors\onedrive,

   <installDIR>\connectors\sponline to <installDIR>\connectors\amzons3,

   <installDIR>\connectors\onedrive, <installDIR>\connectors\sponline

6. If you have configured proxy settings for Box, then you need to copy proxy.properties stored on the old collector at following locations and paste it on the new collector. '<DataDIR>\console\cloud' to <DataDIR>\console\cloud

Migrating Data Insight Management Server

1. Deploy new Data Insight Management Server by referring to Data Insight Installation guide and make sure that you have configured a new Data Insight Collector.
2. Remote desktop to new Data Insight Management server and stop all the Data Insight services on the new Data Insight Management server.
3. Remote desktop to old Data Insight Management server and stop all the Data Insight services on the old Data Insight Management server.
4. Copy entire Data Insight data directory from old Data Insight Management server and paste it on new Management server. Default data directory location is C:\DataInsight\data.
5. Copy existing content stored at …\DataInsight\data directory from old Management server and paste it under new location directory hierarchy under …\DataInsight\data on the new Management server.
6. After pasting all the data, remote desktop to new Management server.
7. Update the nodename.conf file with new Data Insight Management server. Default location of this file is C:\DataInsight\data\conf.
8. Open the file and remove the FQDN of old Data Insight Management server present in the nodename.conf with FQDN of the new Data Insight Management server.
9. Save the changes and close the file.
10. Update FQDN of new Data Insight Management server in nodes table of config.db from default location C:\DataInsight\data\conf. Refer to the latest version of "config.db" to update using sqlite3.exe from command prompt given in step 13.
11. Launch command prompt on the Data Insight Management server
12. From command prompt, go to bin directory of the Data Insight. Default location is C:\Program Files\DataInsight\bin.
13. Enter below commands to connect to config.db and update the FQDN of new Data Insight Management server using the update command of sqlite:
    
14. After updating FQDN, start all the Data Insight services on the new Data Insight Management Server.
15. Once services are up and running, confirm it by navigating to Settings > Data Insight Servers on Data Insight Console.
16. If the health color of all the Data Insight servers is Green, you can assume that all Data Insight servers are properly communicating with Data Insight Management server.
17. Navigating to Settings > Filers on Data Insight Console.
18. If the health color of all devices or filers is Green, you can assume that all devices or filers are properly communicating with their respective Data Insight Collector and Data Insight Indexer nodes.
19. If you have assigned new IP and FQDN to new Management server, you need to change redirect url mentioned in Azure application. This will help in restoring communication of SharePointOnline and OneDrive cloud devices. In addition to this, you need to change redirect url mentioned in Azure Active Directory application. Default URL to access Azure application is https://portal.azure.com. Execute instruction #19 from this document only if you have configured cloud devices in Data Insight.
20. After this point, you can resume Scanning and Auditing jobs for all the configured devices across all the Data Insight Collectors.
21. Turn on IndexWriterJob across all Data Insight Indexer nodes by navigating to Settings > Data Insight Servers.

    • On this page, click Indexer > Jobs

    • By using the available filter option, search IndexWriterJob.

    • Click Select action and click Enable.

    • Repeat this procedure for all Indexer nodes.

22. Turn on ActivityIndexJob across all Data Insight Indexer nodes by navigating to Settings > Data Insight Servers.

    • On this page, click Indexer > Jobs

    • By using the available filter option, search ActivityIndexJob.

    • Click Select action and click Enable.

    • Repeat this procedure for all Indexer nodes.

23. Reconfigure schedule of Summary Dashboard report as per your requirement.

Removing the Old Data Insight Collector and Data Insight Indexer nodes

1. Delete old Collector and Indexer nodes from Data Insight Console by navigating to Settings > Data Insight Servers.
2. From the list, identify old Data Insight collector and Data Insight indexer nodes.
3. Select one server at a time, click Select action and click Delete from the list of menu items.