Arctera Data Insight Administrator's Guide

Last Published:
Product(s): Data Insight (7.1)
Platform: Windows
  1. Section I. Getting started
    1. Introduction to Arctera Data Insight administration
      1. About Arctera Data Insight administration
        1.  
          Operation icons on the Management Console
        2.  
          Data Insight administration tasks
    2. Configuring Data Insight global settings
      1.  
        About Data Insight licensing
      2.  
        SQLite WAL mode
      3.  
        Configuring SMTP server settings
      4. About scanning and event monitoring
        1. Configuring scanning and event monitoring
          1.  
            Considerations for running a parallel scan
      5.  
        Monitoring Indexer Node Storage Utilization
      6. About filtering certain accounts, IP addresses, and paths
        1.  
          About exclude rules for access events
        2.  
          About exclude rules for Scanner
        3. Adding exclude rules to Data Insight
          1.  
            Add/Edit Exclude rule for access events options
          2.  
            Add/Edit Exclude rule for Scanner options
      7. About archiving data
        1.  
          About purging data
        2.  
          Configuring data retention settings
        3.  
          Parameterized Purging of access data using Data Retention
        4.  
          Purging indexes by date/whitespace for deleted files
      8. About Data Insight integration with Data Loss Prevention (DLP)
        1.  
          About configuring Data Insight to integrate with Data Loss Prevention (DLP)
        2.  
          Configuring Data Loss Prevention settings
        3.  
          Importing SSL certificate from the DLP Enforce Server to Data Insight Management Server
        4.  
          About Data Loss Prevention (DLP) integration with Data Insight
      9.  
        Importing sensitive files information through CSV
      10. Configuring advanced analytics
        1.  
          Choosing custom attributes for advanced analytics
      11. About open shares
        1.  
          Configuring an open share policy
      12. About user risk score
        1.  
          User risk weight configuration
      13.  
        Configuring file groups
      14.  
        Configuring Workspace data owner policy
      15.  
        Configuring Management Console settings
      16. About bulk assignment of custodians
        1.  
          Assigning custodians in bulk using a CSV file
        2.  
          Assigning custodians based on data ownership
      17.  
        Configuring Watchlist settings
      18. Configuring Metadata Framework
        1.  
          Using the metadata framework for classification and remediation
      19.  
        Proof of concept
  2. Section II. Configuring Data Insight
    1. Configuring Data Insight product users
      1.  
        About Data Insight users and roles
      2.  
        Reviewing current users and privileges
      3. Adding a user
        1.  
          Add or edit Data Insight user options
      4.  
        Editing users
      5.  
        Deleting users
      6.  
        Configuring authorization for Data Loss Prevention users
      7.  
        Configuring single sign-on (SSO) using security assertion markup language (SAML)
      8.  
        Disabling single sign-on (SSO)
    2. Configuring Data Insight product servers
      1.  
        About Data Insight product servers
      2.  
        Adding a new Data Insight server
      3.  
        Managing Data Insight product servers
      4.  
        Viewing Data Insight server details
      5. About node templates
        1.  
          Managing node templates
        2.  
          Adding or editing node templates
      6.  
        Adding Portal role to a Data Insight server
      7.  
        Adding Classification Server role to a Data Insight server
      8.  
        Assigning Classification Server to a Collector
      9.  
        Associating a Classification Server pool to a Collector
      10.  
        Viewing in-progress scans
      11.  
        Configuring Data Insight services
      12.  
        Configuring advanced settings
      13.  
        Monitoring Data Insight jobs
      14.  
        Rotating the encryption keys
      15.  
        Viewing Data Insight server statistics
      16. About automated alerts for patches and upgrades
        1.  
          Viewing and installing recommended upgrades and patches
      17.  
        Deploying upgrades and patches remotely
      18.  
        Using the Upload Manager utility
      19.  
        About migrating storage devices across Indexers
      20.  
        Viewing the status of a remote installation
    3. Configuring saved credentials
      1. About saved credentials
        1.  
          Managing saved credentials
      2.  
        Handling changes in account password
      3.  
        Data Insight Hash Utility
    4. Configuring directory service domains
      1.  
        About directory domain scans
      2. Adding a directory service domain to Data Insight
        1.  
          Add/Edit Active Directory options
        2.  
          Add/Edit LDAP domain options
        3.  
          Add/Edit NIS domain options
        4.  
          Add/Edit NIS+ domain options
        5. Add/Edit Azure active directory service
          1.  
            Prerequisites for configuring Azure AD
          2.  
            Registering Data Insight with Microsoft to scan Azure AD
          3.  
            Configuring application without user impersonation for Microsoft 365
          4.  
            Creating an application in the Microsoft Azure portal
      3.  
        Managing directory service domains
      4.  
        Fetching users and groups data from NIS+ scanner
      5.  
        Configuring attributes for advanced analytics
      6.  
        Deleting directory service domains
      7.  
        Scheduling scans
      8.  
        Configuring business unit mappings
      9.  
        Importing additional attributes for users and user groups
    5. Configuring containers
      1.  
        About containers
      2. Adding containers
        1.  
          Add new container/Edit container options
      3.  
        Managing containers
  3. Section III. Configuring native file systems in Data Insight
    1. Configuring NetApp 7-mode file server monitoring
      1.  
        About configuring NetApp file server monitoring
      2.  
        Prerequisites for configuring NetApp file servers
      3.  
        Credentials required for configuring NetApp filers
      4.  
        Credentials required for configuring NetApp NFS filers
      5.  
        Configuring SMB signing
      6.  
        About FPolicy
      7.  
        Preparing Data Insight for FPolicy
      8.  
        Preparing the NetApp filer for Fpolicy
      9.  
        Preparing the NetApp vfiler for Fpolicy
      10.  
        Configuring NetApp audit settings for performance improvement
      11.  
        Preparing a non-administrator domain user on the NetApp filer for Data Insight
      12.  
        Enabling export of NFS shares on a NetApp file server
      13.  
        Excluding volumes on a NetApp file server
      14.  
        Handling NetApp home directories in Data Insight
    2. Configuring clustered NetApp file server monitoring
      1.  
        About configuring a clustered NetApp file server
      2.  
        About configuring FPolicy in Cluster-Mode
      3.  
        Pre-requisites for configuring clustered NetApp file servers
      4.  
        Credentials required for configuring a clustered NetApp file server
      5.  
        Preparing a non-administrator local user on the clustered NetApp filer
      6.  
        Preparing a non-administrator domain user on a NetApp cluster for Data Insight
      7.  
        Persistent Store
      8.  
        Preparing Data Insight for FPolicy in NetApp Cluster-Mode
      9.  
        Preparing the ONTAP cluster for FPolicy
      10. About configuring secure communication between Data Insight and cluster-mode NetApp devices
        1.  
          Generating SSL certificates for NetApp cluster-mode authentication
        2.  
          Preparing the NetApp cluster for SSL authentication
      11.  
        Enabling export of NFS shares on a NetApp Cluster-Mode file server
      12.  
        Enabling SSL support for Cluster Mode NetApp auditing
    3. Configuring EMC Celerra or VNX monitoring
      1. About configuring EMC Celerra or VNX filers
        1.  
          About EMC Common Event Enabler (CEE)
        2.  
          Preparing the EMC filer for CEPA
        3.  
          Preparing Data Insight to receive event notification
      2.  
        Credentials required for configuring EMC Celerra filers
    4. Configuring EMC Isilon monitoring
      1.  
        About configuring EMC Isilon filers
      2.  
        Prerequisites for configuration of Isilon or Unity VSA file server monitoring
      3.  
        Credentials required for configuring an EMC Isilon cluster
      4.  
        Configuring audit settings on EMC Isilon cluster using OneFS GUI console
      5.  
        Configuring audit settings on EMC Isilon cluster using the OneFS CLI
      6.  
        Configuring Isilon audit settings for performance improvement
      7.  
        Preparing Arctera Data Insight to receive event notifications from an EMC Isilon or Unity VSA cluster
      8.  
        Creating a non-administrator user for an EMC Isilon cluster
      9.  
        Utilizing access zone's SmartConnect Zone/Alias mappings
      10.  
        Purging the audit logs in an Isilon filer
    5. Configuring EMC Unity VSA file servers
      1.  
        About configuring Dell EMC Unity storage platform
      2.  
        Credentials required for configuring an EMC Unity VSA file server
      3.  
        Configuring audit settings on EMC Unity cluster using Unisphere VSA Unity console
    6. Configuring Hitachi NAS file server monitoring
      1.  
        About configuring Hitachi NAS
      2.  
        Credentials required for configuring a Hitachi NAS EVS
      3.  
        Creating a domain user on a Hitachi NAS file server for Data Insight
      4.  
        Preparing a Hitachi NAS file server for file system auditing
      5.  
        Advanced configuration parameters for Hitachi NAS
    7. Configuring Windows File Server monitoring
      1.  
        About configuring Windows file server monitoring
      2.  
        Credentials required for configuring Windows File Servers
      3.  
        Using the installcli.exe utility to configure multiple Windows file servers
      4.  
        Upgrading the Windows File Server agent
    8. Configuring Veritas File System (VxFS) file server monitoring
      1.  
        About configuring Veritas File System (VxFS) file servers
      2.  
        Credentials required for configuring Veritas File System (VxFS) servers
      3.  
        Enabling export of UNIX/Linux NFS shares on VxFS filers
    9. Configuring monitoring of a generic device
      1.  
        About configuring a generic device
      2.  
        Credentials required for scanning a generic device
    10. Managing file servers
      1.  
        About configuring filers
      2.  
        Viewing configured filers
      3. Adding filers
        1.  
          Add/Edit NetApp filer options
        2.  
          Add/Edit NetApp cluster file server options
        3.  
          Add/Edit EMC Celerra filer options
        4.  
          Add/Edit EMC Isilon file server options
        5.  
          Add/Edit EMC Unity VSA file server options
        6.  
          Add/Edit Windows File Server options
        7.  
          Add/Edit Veritas File System server options
        8.  
          Add/Edit a generic storage device options
        9.  
          Add/Edit Hitachi NAS file server options
      4.  
        Custom schedule options
      5.  
        Editing filer configuration
      6.  
        Deleting filers
      7.  
        Viewing performance statistics for file servers
      8.  
        About disabled shares
      9. Adding shares
        1.  
          Add New Share/Edit Share options
      10.  
        Managing shares
      11.  
        Editing share configuration
      12.  
        Deleting shares
      13.  
        About configuring a DFS target
      14.  
        Adding a configuration attribute for devices
      15.  
        Configuring a DFS target
      16.  
        About the DFS utility
      17.  
        Running the DFS utility
      18.  
        Importing DFS mapping
    11. Renaming storage devices
      1.  
        About renaming a storage device
      2.  
        Viewing the device rename status
      3.  
        Considerations for renaming a storage device
  4. Section IV. Configuring SharePoint data sources
    1. Configuring monitoring of SharePoint web applications
      1.  
        About SharePoint server monitoring
      2.  
        Credentials required for configuring SharePoint servers
      3.  
        Configuring a web application policy
      4. About the Data Insight web service for SharePoint
        1.  
          Installing the Data Insight web service for SharePoint
      5.  
        Viewing configured SharePoint data sources
      6. Adding web applications
        1.  
          Add/Edit web application options
      7.  
        Editing web applications
      8.  
        Deleting web applications
      9. Adding site collections
        1.  
          Add/Edit site collection options
      10.  
        Managing site collections
      11.  
        Removing a configured web application
    2. Configuring monitoring of SharePoint Online accounts
      1. About SharePoint Online account monitoring
        1.  
          Prerequisites for configuring SharePoint Online account
      2.  
        Configuring user with minimum privileges in Microsoft 365
      3.  
        Creating an application in the Microsoft Azure portal
      4.  
        Configuring application without user impersonation for Microsoft 365
      5.  
        Adding SharePoint Online accounts
      6.  
        Managing a SharePoint Online account
      7. Adding site collections to SharePoint Online accounts
        1.  
          Add/Edit site collection options
      8.  
        Managing site collections
  5. Section V. Configuring cloud data sources
    1. Configuring monitoring of Box accounts
      1.  
        About configuring Box monitoring
      2.  
        Using a co-admin account to monitor Box resources
      3. Configuring monitoring of cloud sources in Data Insight
        1.  
          Add/Edit Box account
      4.  
        Configuring Box cloud resources through proxy server
      5.  
        Data Insight limitations for Box permissions
    2. Configuring OneDrive account monitoring
      1.  
        About configuring OneDrive monitoring
      2.  
        Configuring user with minimum privileges in Microsoft 365
      3.  
        Creating an application in the Microsoft Azure portal
      4.  
        Configuring application without user impersonation for Microsoft 365
      5.  
        Add/Edit OneDrive account
      6. Adding OneDrive cloud accounts
        1.  
          Add/edit OneDrive user accounts
    3. Configuring Azure Netapp Files Device
      1.  
        About Azure Netapp Files
      2.  
        Configuring Azure Netapp Files Device
    4. Managing cloud sources
      1.  
        Viewing configured cloud sources
      2.  
        Managing cloud sources
  6. Section VI. Configuring Object Storage Sources
    1. Amazon S3
      1.  
        About Amazon Simple Storage Service (Amazon S3)
      2.  
        Configuring Amazon S3 account monitoring
      3.  
        Configuring Audit Events in AWS
      4.  
        Creating an Athena table
      5.  
        Adding Amazon S3 account
      6.  
        Limitations for Amazon S3 in Data Insight
      7. Managing Amazon S3 data source
        1.  
          Monitored Buckets
        2.  
          Classification
  7. Section VII. Health and monitoring
    1. Using Arctera Data Insight dashboards
      1.  
        Viewing the system health overview
      2.  
        Viewing the scanning overview
      3.  
        Viewing the scan status of storage devices
      4.  
        Viewing the scan history of storage devices
    2. Monitoring Data Insight
      1.  
        Viewing events
      2.  
        About high availability notifications
      3.  
        Monitoring the performance of Data Insight servers
      4.  
        Configuring email notifications
      5.  
        Enabling Windows event logging
      6.  
        Viewing scan errors
  8. Section VIII. Alerts and policies
    1. Configuring policies
      1.  
        About Data Insight policies
      2. Managing policies
        1.  
          Create Data Activity Trigger policy options
        2.  
          Create User Activity Deviation policy options
        3.  
          Create Real-time Permitted User-based Activity Policy options
        4.  
          Create Real-time Restricted User-based Activity Policy options
        5.  
          Create Real-time Sensitive Data Activity policy options
      3.  
        Managing alerts
  9. Section IX. Remediation
    1. Configuring remediation settings
      1. About configuring permission remediation
        1.  
          Managing and configuring permission remediation
        2.  
          Configuring exclusions for permission recommendation
      2.  
        About managing data
      3.  
        About deleting files
      4. About configuring archive options for Enterprise Vault
        1.  
          Adding new Enterprise Vault servers
        2.  
          Managing Enterprise Vault servers
        3.  
          Mapping file server host names
      5. Configuring Microsoft Purview Information Protection (MIP) Label
        1.  
          Removing MIP Label
      6.  
        Using custom scripts to manage data
      7.  
        Viewing and managing the status of an operation
  10. Section X. Reference
    1. Appendix A.  Data Insight best practices
      1.  
        Understanding Data Insight best practices
    2. Appendix B.  Migrating Data Insight components
      1.  
        Migrating Data Insight components
    3. Appendix C. Backing up and restoring data
      1.  
        Selecting the backup and restore order
      2.  
        Backing up and restoring the Data Insight Management Server
      3.  
        Backing up and restoring the Indexer node
      4.  
        Understanding Data Insight best practices
    4. Appendix D. Data Insight health checks
      1. About Data Insight health checks
        1.  
          Services checks
        2.  
          Deployment details checks
        3.  
          Generic checks
        4.  
          Data Insight Management Server checks
        5.  
          Data Insight Indexer checks
        6.  
          Data Insight Collector checks
        7.  
          Data Insight Windows File Server checks
        8.  
          Data Insight SharePoint checks
        9.  
          Classification server health checks
        10.  
          Data Insight self service portal server health checks
    5. Appendix E. Command File Reference
      1.  
        fg.exe
      2.  
        indexcli.exe
      3.  
        reportcli.exe
      4.  
        scancli.exe
      5.  
        installcli.exe
    6. Appendix F. Data Insight jobs
      1.  
        Scheduled Data Insight jobs
    7. Appendix G. Troubleshooting
      1.  
        About general troubleshooting procedures
      2.  
        About the Health Audit report
      3.  
        Location of Data Insight logs
      4.  
        Downloading Data Insight logs
      5.  
        Migrating the data directory to a new location
      6. Troubleshooting FPolicy issues on NetApp devices
        1.  
          Viewing FPolicy-related errors and warnings
        2.  
          Resolving FPolicy connection issues
      7.  
        Troubleshooting EMC Celera or VNX configuration issues
      8.  
        Troubleshooting EMC Isilon configuration issues
      9.  
        Troubleshooting SharePoint configuration issues
      10.  
        Troubleshooting Hitachi NAS configuration issues
      11.  
        Troubleshooting installation of Tesseract software
      12.  
        Troubleshooting RHEL 9 upgrade issue

Configuring scanning and event monitoring

You can configure Data Insight to globally turn on or off receipt of event notifications and safeguards related to FPolicy communication.

Data Insight allows you to disable scanning of all file systems. The scans can be enabled at any convenient time.

You can also configure whether you want the Scanner to fetch the Access Control Lists (ACLs) defined on folders and ownership information for files and folders. For Windows File Servers, you can configure safeguards that prevent an agent from taking up too much disk space.

To configure scanning and event monitoring

  1. In the Management Console, click Settings > Scanning and Event Monitoring.

    You can view the state of scanning and event monitoring.

  2. To change the state of a process to Enabled or Disabled, click Edit.
  3. Do one of the following:

    • Select the check box for the process that you want to enable.

    • Clear the check box for the process that you want to disable.

  4. Click Save to save the changes.

Table: Scanning and Event Monitoring options

Option

Description

Scan File System meta-data

Clear the check box to turn off all future file system scanning on all filers. Once you save the setting, it will also stop all currently running scans.

Get Folder ACLs

Clear the check box if you do not want Scanner to fetch Access Control List (ACLs) for folders during scanning.

If you disable this option, the Workspace > Permissions tab in the Console is disabled and permission related reports will not produce any data. If you do not need permissions data, you can disable this option to make the scans run faster.

Get Ownership information for files and folders

Clear the check box if you do not want Scanner to fetch the Owner attribute for files and folders.

Ownership information is used to determine ownership for data when access events are not available. If you do not need this information, you can disable this option to make scans run faster.

Throttling for NetApp filers

Select Throttle scanning based on latency of the filer to enable throttling of Data Insight scans for NetApp 7-mode and Cluster-Mode file servers. This option is not selected by default.

Data Insight collects latency information from NetApp file servers. It can use this information to throttle scanning, if latency of the file server increases above a certain level. This ensures scanner does not put additional load on the file server during peak load conditions.

You can configure the following parameters to enable throttling for NetApp file servers:

  • Latency threshold - Specify latency in milliseconds, which when crossed, should throttle scanning for the file server.

  • Minimum pause - Specify the minimum duration (in milliseconds) for which the scanner should pause between paths when in throttling mode.

  • Back off value - If increased latency is sustained, pause interval will be increased by the Back off value specified (in milliseconds).

  • Maximum pause - Specify the maximum pause interval for the scanner (in milliseconds). If exceeded, pause interval is no longer incremented by Back off value.

Monitor file system access events

Clear the check box to stop Data Insight from collecting access events from all file servers. In case of NetApp, it means all collector nodes will disconnect their Fpolicy connections to file servers.

Disk safeguard settings for Windows File Server agents

Select Enable node safeguard check box to monitor the disk usage on the Windows File Server node, and to prevent it from running out of disk space by implementing safeguards.

You can specify the threshold for disk utilization in terms of percentage and size. The DataInsightWatchdog service initiates the safeguard mode for the Windows File Server node if the free disk space falls under the configured thresholds.

The DataInsightWatchdog service automatically resets the safeguard mode when the free disk space is more than the configured thresholds.

You can edit the threshold limits as required. If you specify values in terms of both percentage and size, then the condition that is fulfilled first is applied to initiate the safeguard mode.

Note:

It is recommended that you configure the same safeguard thresholds for the Windows File Server and the Classification Server (if present in your environment).

For more information about Configuring classification, see the Data Insight Classification Guide

For more information on the DataInsight Watchdog service, see the Arctera Data Insight Installation Guide.

Fpolicy Safeguard settings

Select the following:

  • Enable FPolicy 7-Mode Safeguard to initiate safeguard mode for the NetApp file servers.

  • Select Enable FPolicy 7-Mode safeguard for VFilers to initiate safeguard mode for virtual file servers by monitoring latency statistics for the associated physical NetApp filers.

    The safeguard mode for the virtual filers is initiated only if the details of the physical filer corresponding to the virtual filer are provided while adding the virtual filer.

  • Select Enable FPolicy Cluster Mode Safeguard to initiate safeguard mode for the NetApp cluster mode file servers.

This Fpolicy Safeguard settings are not selected by default.

Data Insight collects latency information from NetApp file servers. It can use this information to initiate safeguard mode, if latency of the file server increases above or falls below a certain level. When the safeguard is in effect, Data Insight drops its Fpolicy connection to the filer. This ensures event collection does not put additional load on the file server in peak load conditions.

If the latency (CIFS or NFS) on the physical file server increases above the configured threshold, Data Insight disconnects from the associated virtual file server even if monitoring for that type (CIFS/NFS) was not enabled on the virtual filer. This information is also displayed on the Data Insight System Overview dashboard.

Configure the following values:

  • The high and low thresholds for CIFS and NFS.

  • The number of samples to be considered to calculate the average latency.

  • The minimum time to wait before Data Insight reconnects to the filer after a disconnection.

See Add/Edit NetApp filer options.

You can choose to override the global scan throttling and FPolicy safeguard settings for specific Collector nodes.

See Configuring advanced settings.