Veritas Data Insight Administrator's Guide

Last Published:
Product(s): Data Insight (6.1.5)
  1. Section I. Getting started
    1. Introduction to Veritas Data Insight administration
      1. About Veritas Data Insight administration
        1.  
          Operation icons on the Management Console
        2.  
          Data Insight administration tasks
        3.  
          Supported data sources and platforms
    2. Configuring Data Insight global settings
      1. Overview of Data Insight licensing
        1.  
          Managing Data Insight licenses
      2.  
        Configuring SMTP server settings
      3. About scanning and event monitoring
        1. Configuring scanning and event monitoring
          1.  
            Considerations for running a parallel scan
      4. About filtering certain accounts, IP addresses, and paths
        1.  
          About exclude rules for access events
        2.  
          About exclude rules for Scanner
        3. Adding exclude rules to Data Insight
          1.  
            Add/Edit Exclude rule for access events options
          2.  
            Add/Edit Exclude rule for Scanner options
      5. About archiving data
        1.  
          About purging data
        2.  
          Configuring data retention settings
      6. About Data Insight integration with Symantec Data Loss Prevention (DLP)
        1.  
          About configuring Data Insight to integrate with Data Loss Prevention (DLP)
        2.  
          Configuring Symantec Data Loss Prevention settings
        3.  
          Importing SSL certificate from the DLP Enforce Server to Data Insight Management Server
        4.  
          About Symantec Data Loss Prevention (DLP) integration with Data Insight
      7.  
        Importing sensitive files information through CSV
      8. Configuring advanced analytics
        1.  
          Choosing custom attributes for advanced analytics
      9. About open shares
        1.  
          Configuring an open share policy
      10.  
        Configuring file groups
      11.  
        Configuring Workspace data owner policy
      12.  
        Configuring Management Console settings
      13. About bulk assignment of custodians
        1.  
          Assigning custodians in bulk using a CSV file
        2.  
          Assigning custodians based on data ownership
      14.  
        Configuring Watchlist settings
  2. Section II. Configuring Data Insight
    1. Configuring Data Insight product users
      1.  
        About Data Insight users and roles
      2.  
        Reviewing current users and privileges
      3. Adding a user
        1.  
          Add or edit Data Insight user options
      4.  
        Editing users
      5.  
        Deleting users
      6.  
        Configuring authorization for Symantec Data Loss Prevention users
    2. Configuring Data Insight product servers
      1.  
        About Data Insight product servers
      2.  
        Adding a new Data Insight server
      3.  
        Managing Data Insight product servers
      4.  
        Viewing Data Insight server details
      5. About node templates
        1.  
          Managing node templates
        2.  
          Adding or editing node templates
      6.  
        Adding Portal role to a Data Insight server
      7.  
        Adding Classification Server role to a Data Insight server
      8.  
        Assigning Classification Server to a Collector
      9.  
        Associating a Classification Server pool to a Collector
      10.  
        Viewing in-progress scans
      11.  
        Configuring Data Insight services
      12.  
        Configuring advanced settings
      13.  
        Monitoring Data Insight jobs
      14.  
        Rotating the encryption keys
      15.  
        Viewing Data Insight server statistics
      16. About automated alerts for patches and upgrades
        1.  
          Viewing and installing recommended upgrades and patches
      17.  
        Deploying upgrades and patches remotely
      18.  
        Using the Upload Manager utility
      19.  
        About migrating storage devices across Indexers
      20.  
        Viewing the status of a remote installation
    3. Configuring saved credentials
      1. About saved credentials
        1.  
          Managing saved credentials
      2.  
        Handling changes in account password
    4. Configuring directory service domains
      1.  
        Fetching users and groups data from NIS+ scanner
      2.  
        Configuring attributes for advanced analytics
      3.  
        Deleting directory service domains
      4.  
        Scheduling scans
      5.  
        Configuring business unit mappings
      6.  
        Importing additional attributes for users and user groups
    5. Configuring containers
      1.  
        About containers
      2. Adding containers
        1.  
          Add new container/Edit container options
      3.  
        Managing containers
  3. Section III. Configuring native file systems in Data Insight
    1. Configuring NetApp file server monitoring
      1.  
        About configuring NetApp file server monitoring
      2.  
        Prerequisites for configuring NetApp file servers
      3.  
        Credentials required for configuring NetApp filers
      4.  
        Credentials required for configuring NetApp NFS filers
      5.  
        Configuring SMB signing
      6.  
        About FPolicy
      7.  
        Preparing Data Insight for FPolicy
      8.  
        Preparing the NetApp filer for Fpolicy
      9.  
        Preparing the NetApp vfiler for Fpolicy
      10.  
        Configuring NetApp audit settings for performance improvement
      11.  
        Preparing a non-administrator domain user on the NetApp filer for Data Insight
      12.  
        Enabling export of NFS shares on a NetApp file server
      13.  
        Excluding volumes on a NetApp file server
      14.  
        Handling NetApp home directories in Data Insight
    2. Configuring clustered NetApp file server monitoring
      1.  
        About configuring a clustered NetApp file server
      2.  
        About configuring FPolicy in Cluster-Mode
      3.  
        Pre-requisites for configuring clustered NetApp file servers
      4.  
        Credentials required for configuring a clustered NetApp file server
      5.  
        Preparing a non-administrator local user on the clustered NetApp filer
      6.  
        Preparing a non-administrator domain user on a NetApp cluster for Data Insight
      7.  
        Preparing Data Insight for FPolicy in NetApp Cluster-Mode
      8.  
        Preparing the ONTAP cluster for FPolicy
      9. About configuring secure communication between Data Insight and cluster-mode NetApp devices
        1.  
          Generating SSL certificates for NetApp cluster-mode authentication
        2.  
          Preparing the NetApp cluster for SSL authentication
      10.  
        Enabling export of NFS shares on a NetApp Cluster-Mode file server
    3. Configuring EMC Celerra or VNX monitoring
      1. About configuring EMC Celerra or VNX filers
        1.  
          About EMC Common Event Enabler (CEE)
        2.  
          Preparing the EMC filer for CEPA
        3.  
          Preparing Data Insight to receive event notification
      2.  
        Credentials required for configuring EMC Celerra filers
    4. Configuring EMC Isilon monitoring
      1.  
        About configuring EMC Isilon filers
      2.  
        Prerequisites for configuration of Isilon or Unity VSA file server monitoring
      3.  
        Credentials required for configuring an EMC Isilon cluster
      4.  
        Configuring audit settings on EMC Isilon cluster using OneFS GUI console
      5.  
        Configuring audit settings on EMC Isilon cluster using the OneFS CLI
      6.  
        Configuring Isilon audit settings for performance improvement
      7.  
        Preparing Veritas Data Insight to receive event notifications from an EMC Isilon or Unity VSA cluster
      8.  
        Creating a non-administrator user for an EMC Isilon cluster
      9.  
        Using SmartConnect mapping for access zones
      10.  
        Purging the audit logs in an Isilon filer
    5. Configuring EMC Unity VSA file servers
      1.  
        About configuring Dell EMC Unity storage platform
      2.  
        Credentials required for configuring an EMC Unity VSA file server
      3.  
        Configuring audit settings on EMC Unity cluster using Unisphere VSA Unity console
    6. Configuring Hitachi NAS file server monitoring
      1.  
        About configuring Hitachi NAS
      2.  
        Credentials required for configuring a Hitachi NAS EVS
      3.  
        Creating a domain user on a Hitachi NAS file server for Data Insight
      4.  
        Preparing a Hitachi NAS file server for file system auditing
      5.  
        Advanced configuration parameters for Hitachi NAS
    7. Configuring Windows File Server monitoring
      1.  
        About configuring Windows file server monitoring
      2.  
        Credentials required for configuring Windows File Servers
      3.  
        Using the installcli.exe utility to configure multiple Windows file servers
      4.  
        Upgrading the Windows File Server agent
    8. Configuring Veritas File System (VxFS) file server monitoring
      1.  
        About configuring Veritas File System (VxFS) file servers
      2.  
        Credentials required for configuring Veritas File System (VxFS) servers
      3.  
        Enabling export of UNIX/Linux NFS shares on VxFS filers
    9. Configuring monitoring of a generic device
      1.  
        About configuring a generic device
      2.  
        Credentials required for scanning a generic device
    10. Managing file servers
      1.  
        About configuring filers
      2.  
        Viewing configured filers
      3. Adding filers
        1.  
          Add/Edit NetApp filer options
        2.  
          Add/Edit NetApp cluster file server options
        3.  
          Add/Edit EMC Celerra filer options
        4.  
          Add/Edit EMC Isilon file server options
        5.  
          Add/Edit EMC Unity VSA file server options
        6.  
          Add/Edit Windows File Server options
        7.  
          Add/Edit Veritas File System server options
        8.  
          Add/Edit a generic storage device options
        9.  
          Add/Edit Hitachi NAS file server options
      4.  
        Custom schedule options
      5.  
        Editing filer configuration
      6.  
        Deleting filers
      7.  
        Viewing performance statistics for file servers
      8.  
        About disabled shares
      9. Adding shares
        1.  
          Add New Share/Edit Share options
      10.  
        Managing shares
      11.  
        Editing share configuration
      12.  
        Deleting shares
      13.  
        About configuring a DFS target
      14.  
        Adding a configuration attribute for devices
      15.  
        Configuring a DFS target
      16.  
        About the DFS utility
      17.  
        Running the DFS utility
      18.  
        Importing DFS mapping
    11. Renaming storage devices
      1.  
        About renaming a storage device
      2.  
        Viewing the device rename status
      3.  
        Considerations for renaming a storage device
  4. Section IV. Configuring SharePoint data sources
    1. Configuring monitoring of SharePoint web applications
      1.  
        About SharePoint server monitoring
      2.  
        Credentials required for configuring SharePoint servers
      3.  
        Configuring a web application policy
      4. About the Data Insight web service for SharePoint
        1.  
          Installing the Data Insight web service for SharePoint
      5. Adding web applications
        1.  
          Add/Edit web application options
      6.  
        Editing web applications
      7.  
        Deleting web applications
      8. Adding site collections
        1.  
          Add/Edit site collection options
      9.  
        Managing site collections
      10.  
        Removing a configured web application
    2. Configuring monitoring of SharePoint Online accounts
      1. About SharePoint Online account monitoring
        1.  
          Prerequisites for configuring SharePoint Online account
      2.  
        Registering Data Insight with Microsoft to enable SharePoint Online account monitoring
      3.  
        Configuring an administrator account for Data Insight
      4. Adding SharePoint Online accounts
        1.  
          Add/Edit SharePoint Online account options
      5.  
        Managing a SharePoint Online account
      6. Adding site collections to SharePoint Online accounts
        1.  
          Add/Edit site collection options
      7.  
        Managing site collections
  5. Section V. Configuring cloud data sources
    1. Configuring monitoring of Box accounts
      1.  
        About configuring Box monitoring
      2.  
        Using a co-admin account to monitor Box resources
      3. Configuring monitoring of cloud sources in Data Insight
        1.  
          Add/Edit Box account
      4.  
        Configuring Box cloud resources through proxy server
    2. Configuring OneDrive account monitoring
      1.  
        About configuring OneDrive monitoring
      2.  
        Registering Data Insight with Microsoft to enable OneDrive account monitoring
      3.  
        Configuring user impersonation in Office 365
      4.  
        Add/Edit OneDrive account
      5. Adding OneDrive user accounts
        1.  
          Add/edit OneDrive user accounts
    3. Managing cloud sources
      1.  
        Viewing configured cloud sources
      2.  
        Managing cloud sources
  6. Section VI. Configuring ECM data sources
    1. Configuring Documentum data source
      1.  
        About Documentum device monitoring
      2.  
        Credentials required for configuring Documentum devices
      3. Configuring Documentum monitoring in Data Insight
        1.  
          Add/Edit Documentum device
      4.  
        Managing ECM Sources
      5. Adding repositories
        1.  
          Add New/Edit Repository options
      6.  
        Managing repositories
  7. Section VII. Health and monitoring
    1. Using Veritas Data Insight dashboards
      1.  
        Viewing the system health overview
      2.  
        Viewing the scanning overview
      3.  
        Viewing the scan status of storage devices
      4.  
        Viewing the scan history of storage devices
    2. Monitoring Data Insight
      1.  
        Viewing events
      2.  
        About high availability notifications
      3.  
        Monitoring the performance of Data Insight servers
      4.  
        Configuring email notifications
      5.  
        Enabling Windows event logging
      6.  
        Viewing scan errors
  8. Section VIII. Alerts and policies
    1. Configuring policies
      1.  
        About Data Insight policies
      2. Managing policies
        1.  
          Create Data Activity Trigger policy options
        2.  
          Create User Activity Deviation policy options
        3.  
          Create Real-time Data Activity User Whitelist-based policy options
        4.  
          Create Real-time Data Activity User Blacklist-based policy options
        5.  
          Create Real-time Sensitive Data Activity policy options
      3.  
        Managing alerts
  9. Section IX. Remediation
    1. Configuring remediation settings
      1. About configuring permission remediation
        1.  
          Managing and configuring permission remediation
        2.  
          Configuring exclusions for permission recommendation
      2.  
        About managing data
      3. About configuring archive options for Enterprise Vault
        1.  
          Adding new Enterprise Vault servers
        2.  
          Managing Enterprise Vault servers
        3.  
          Mapping file server host names
      4. About deleting files from CIFS devices
        1.  
          Configuring deletion of files and folders
        2.  
          Deleting files from the Workspace tab on the Data Insight interface
        3.  
          Deleting files from the Reports tab on the Data Insight interface
      5.  
  10. Section X. Reference
    1. Appendix A. Backing up and restoring data
      1.  
        Selecting the backup and restore order
      2.  
        Backing up and restoring the Data Insight Management Server
      3.  
        Backing up and restoring the Indexer node
    2. Appendix B. Data Insight health checks
      1. About Data Insight health checks
        1.  
          Services checks
        2.  
          Deployment details checks
        3.  
          Generic checks
        4.  
          Data Insight Management Server checks
        5.  
          Data Insight Indexer checks
        6.  
          Data Insight Collector checks
        7.  
          Data Insight Windows File Server checks
        8.  
          Data Insight SharePoint checks
      2.  
        Understanding Data Insight best practices
    3. Appendix C. Command File Reference
      1.  
        fg.exe
      2.  
        indexcli.exe
      3.  
        reportcli.exe
      4.  
        scancli.exe
      5.  
        installcli.exe
    4. Appendix D. Data Insight jobs
      1.  
        Scheduled Data Insight jobs
    5. Appendix E. Troubleshooting
      1.  
        About general troubleshooting procedures
      2.  
        About the Health Audit report
      3.  
        Location of Data Insight logs
      4.  
        Downloading Data Insight logs
      5.  
        Migrating the data directory to a new location
      6. Troubleshooting FPolicy issues on NetApp devices
        1.  
          Viewing FPolicy-related errors and warnings
        2.  
          Resolving FPolicy connection issues
      7.  
        Troubleshooting EMC Celera or VNX configuration issues
      8.  
        Troubleshooting EMC Isilon configuration issues
      9.  
        Troubleshooting SharePoint configuration issues
      10.  
        Troubleshooting Hitachi NAS configuration issues
      11.  
        Troubleshooting installation of Tesseract software

Scheduled Data Insight jobs

Each Data Insight service performs several actions on a scheduled basis. These services are called jobs. The section explains the function of the important jobs that run in various services. The schedule for few jobs can be changed from the Advanced Settings tab of the Server details page.

Table: Communication service jobs

Job

Description

ADScanJob

Initiates the adcli process on the Management Server to scan the directory servers. Ensure the following:

  • The directory servers are added to the Data Insight configuration.

  • The credentials specified when adding the directory server have permissions to scan the directory server.

CollectorJob

Initiates the collector process to pre-process raw audit events received from storage devices. The job applies exclude rules and heuristics to generate audit files to be sent to the Indexers. It also generates change-logs that are used for incremental scanning.

ChangeLogJob

The CollectorJob generates changelog files containing list of changed paths, one per device, in the changelog folder. There cab be multiple files with different timestamps for each device. The ChangeLogJob merges all changelog files for a device.

ScannerJob

Initiates the scanner process to scan the shares and site collections added to Data Insight.

Creates the scan database for each share that it scanned in the data\outbox folder.

IScannerJob

Intiates the incremental scan process for shares or site-collections for paths that have changed on those devices since the last scan.

CreateWorkflowDBJob

Runs only on the Management Server. It creates the database containing the data for DLP Incident Management, Entitlement Review, and Ownership Confirmation workflows based on the input provided by users.

DlpSensitiveFilesJob

Retrieves policies and sensitive file information from Data Loss Prevention (DLP).

FileTransferJob

Transfers the files from the data\outbox folder from a node to the inbox folder of the appropriate node.

FileTransferJob_content

Runs every 10 seconds on the Windows File Server.

Routes content file and CSQLite file to the assigned Classification Server.

FileTransferJob_Evt

Sends Data Insight events database from the worker node to the Management Server.

FileTransferJob_WF

Transfers workflow files from Management Server to the Portal service.

FileTransferJob_classify

Runs on all Data Insight nodes once every minute.

It distributes the classification events between Data Insight nodes.

IndexWriterJob

Runs on the Indexer node; initiates the idxwriter process to update the Indexer database with scan (incremental and full), tags, and audit data.

After this process runs, you can view newly added or deleted folders and recent access events on shares on the Management Console.

ActivityIndexJob

Runs on the Indexer node; It updates the activity index every time the index for a share or site collection is updated.

The Activity index is used to speed up the computation of ownership of data.

IndexCheckJob

Verifies the integrity of the index databases on an Indexer node.

PingHeartBeatJob

Sends the heartbeat every minute from the worker node to the Data Insight Management Server.

PingMonitorJob

Runs on the Management Server. It monitors the heartbeat from the worker nodes; sends notifications in case it does not get a heartbeat from the worker node.

SystemMonitorJob

Runs on the worker nodes and on the Management Server. Monitors the CPU, memory, and disk space utilization at a scheduled interval. The process sends notifications to the user when the utilization exceeds a certain threshold value.

DiscoverSharesJob

Discovers shares, site collections, or equivalent on the devices for which you have selected the Automatically discover and monitor shares on this filer check box when configuring the device in Data Insight

ScanPauseResumeJob

Checks the changes to the pause and resume settings on the Data Insight servers, and accordingly pauses or resumes scans.

DataRetentionJob

Enforces the data retention policies, which include archiving old index segments and deleting old segments, indexes for deleted objects, old system events, and old alerts.

IndexVoldbJob

Runs on the Management Server and executes the command voldb.exe --index which consumes the device volume utilization information it receives from various Collector nodes.

SendNodeInfoJob

Sends the node information, such as the operating system, and the Data Insight version running on the node to the Management Server. You can view this information on the Data Insight Server > Overview page of the Management Console.

EmailAlertsJob

Runs on the Management Server and sends email notifications as configured in Data Insight.The email notifications pertain to events happening in the product, for example, a directory scan failure. You can view them on the Settings > System Overview page of the Management Console.

LocalUsersScanJob

Runs on the Collector node that monitors configured file servers and SharePoint servers. In case of a Windows File Server that uses agent to monitor access events, it runs on the node on which the agent is installed.

It scans the local users and groups on the storage devices.

UpdateCustodiansJob

Runs on the Indexer node and updates the custodian information in the Data Insight configuration.

CompactJob

Compresses the attic folder and err folders in <datadir>\collector, <datadir>\scanner, and <datadir>\indexer folders. The process uses the Windows compression feature to set the "compression" attribute for the folders.

The job also deletes stale data that's no longer being used.

Compact_Job_Report

Compresses the folders that store report output.

StatsJob

On the Indexer node, it records index size statistics to lstats.db. The information is used to display the filer statistics on the Data Insight Management Console.

MergeStatsJob

Rolls up (into hourly, daily and weekly periods) the published statistics. On the Collector nodes for Windows Filer Server, the job consolidates statistics from the filer nodes.

StatsJob_Index_Size

Publishes statistics related to the size of the index.

StatsJob_Latency

On the Collector node, it records the filer latency statistics for NetApp filers.

SyncScansJob

Gets current scan status from all Collector nodes. The scan status is displayed on the Settings > Scanning Dashboard > In-progress Scans tab of the Management Console.

SPEnableAuditJob

Enables auditing for site collections (within the web application), which have been added to Data Insight for monitoring.

By default, the job runs every 10 minutes.

SPAuditJob

Collects the audit logs from the SQL Server database for a SharePoint web application and generates SharePoint audit databases in Data Insight.

SPScannerJob

Scans the site collections at the scheduled time and fetch data about the document and picture libraries within a site collection and within the sites in the site collection.

NFSUserMappingJob

Maps every UID in raw audit files for NFS and VxFS with an ID generated for use in Data Insight. Or generates an ID corresponding to each User and Group ID in raw audit files received from NFS/VxFS.

MsuAuditJob

Collects statistics information for all indexers on the Indexer.

MsuMigrationJob

Checks whether a filer migration is in process and carries it out.

ProcessEventsJob

Processes all the Data Insight events received from worker nodes and adds them to the yyyy-mm-dd_events.db file on the Management Server.

ProcessEventsJob_SE

Processes scan error files.

SpoolEventsJob

Spools events on worker nodes to be sent to Management Server.

WFStatusMergeJob

Merges the workflow and action status updates for remediation workflows (DLP Incident Remediation, Entitlement Reviews, Ownership Confirmation), Enterprise Vault archiving, and custom actions and update the master workflow database with the details so that users can monitor the progress of workflows and actions from the Management Console.

UpdateConfigJob

Reconfigures jobs based on the configuration changes made on the Management Server.

DeviceAuditJob

Fetches the audit records from the Hitachi NAS EVS that are configured with Data Insight.

By default, this job runs in every 5 seconds.

HNasEnableAuditJob

Enables the Security Access Control Lists (SACLs) for the shares when a Hitachi NAS filer is added.

By default, this job runs in every 10 minutes.

WorkflowActionExecutionJob

This service reads the request file created on the Management Server when a Records Classification workflow is submitted from the Portal. The request file contains the paths on which an Enterprise Vault action is submitted. When the action on the paths is complete, the job updates the request file with the status of the action.

By default, this job runs in every 1 hour.

UserRiskJob

Runs on each Indexer. The job updates hashes used to compute the user risk score.

By default, the job runs at 2:00 A.M. everyday.

UpdateWFCentralAuditDBJob

Runs only on the Management Server. It is used to update the workflow audit information in <DATA_DIR>/workflow/workflow_audit.db.

By default, this job runs every 1 minute.

TagsConsumerJob

Parses the CSV file containing tags for paths. Imports the attributes into Data Insight and creates a Tags database for each filesystem object.

By default, this job runs once every day.

KeyRotationJob

Run the job on demand to change the encryption keys. It is not an automatically scheduled job.

It is recommended to run this job after the Data Insight servers including Windows File Agent server is upgraded to 5.2.

If you want to run the KeyRotationJob without upgrading all the servers, restart all services on the servers that have not been upgraded after the KeyRotationJob is executed and the configuration database is replicated on these servers.

RiskDossierJob

Runs on each Indexer and computes the number of files accessible and number of sensitive files accessible to each user on each share.

This job runs every day at 11.00 P.M. by default.

ClassifyInputJob

Runs every 10 seconds on the Management Server.

The job processes the classification requests from the Data Insight console and from reports for the consumption of the book keeping database.

ClassifyBatchJob

Runs every minute on the Indexer.

The job splits the classification batch input databases for the scanner's consumption, which are later pushed to the Collector.

ClassifyIndexJob

Runs once every minute on the Indexer node.

Updates the index with classification tags and also updates the status of the book keeping database.

ClassifyMergeStatusJob

Runs once every minute on the Management Server.

The job calls the files with the classification update status that are received from each indexer. These files are automatically created on the indexer whenever updates are available. It also updates the global book keeping database that is used to show high level classification status on the Console.

CloudDeviceAuditJob_sponline

Runs once every 70 seconds on the Collector.

Collects the audit data for site collections (within the SharePoint Online account), which have been added to Data Insight for monitoring.

CloudDeviceAuditJob_onedrive

Runs once every 70 seconds on the Collector.

Fetches the audit records for the OneDrive accounts that are configured with Data Insight.

RTWBJob

Runs once every 1 minute on the Indexer to evaluate configured Real-time Data Activity User Whitelist-based and Data Activity User Blacklist-based policies and generates alerts.

The following processes run in the Data Insight WatchDog service

Table: WatchDog service jobs

Job

Description

SyncPerformanceStatsJob -

Runs only on the Management server. Fetches performance related statistics from all other servers.

SystemMonitorJob

Gathers statistics like disk usage, CPU, memory usage.

SystemMonitorJob_backlog

Gathers statistics for unprocessed backlog files.

UpdateConfigJob

Reconfigures its own jobs based on configuration updates from the Management Server.

The following processes run in the Data Insight Workflow service

Table: Workflow service jobs

Job

Description

WFStepExecutorJob

Processes actions for Enterprise Vault archiving, requests for permission remediation, and custom actions configured in Data Insight.

WFStepExecutorJob_im

Processes workflows of type Entitlement Reviews, DLP Incident Remediation, and Ownership confirmation. It also sends email reminders containing links to the remediation portal to the custodians at a specified interval.

UpdateConfigJob

Updates its schedules based on the configuration changes made on the Management Server.

WFSpoolStatusJob

Reads the workflow data every minute, and if there are any new updates in last minute, it creates a status database with the new updates.

FileTransferJob_WF

Transfer workflow status databases from the Self-Service portal nodes to the Management Server.

The following processes run in the Data Insight Webserver service.

Table: Webserver service jobs

Job

Description

CustodianSummaryReportJob

Periodically runs the custodian summary report, which is used to determine the custodians assigned in Data Insight for various resources. The output produced by this report is used in DLP Incident Remediation, Entitlement Review, and Ownership Confirmation workflows.

HealthAuditReportJob

Periodically creates a report summarizing health of the entire deployment, and stores it to log/health_audit folder on the Management Server. The report aids Veritas Support to troubleshoot issues on your setup.

PolicyJob

Evaluates configured policies in the system and raises alerts.

PurgeReportsJob

Deletes older report outputs.

UpdateConfigJob

Updates configuration database on the worker nodes based on the configuration changes made on the Management Server.

UserIndexJob_merge

Consolidates user activity and permission map from all indexers.

UserIndexJob_split

Requests each Indexer for user activity and permission map.

UserRiskMergeJob

This job runs on the Management Server. Its default schedule is 6:00 A.M. every day. The job combines data from all MSUs into a single risk score value for each user. This job creates the userrisk_dashboard.db in the DATA_DIR\conf folder.

The following processes run in the Data Insight Classification service.

Table: Classification service jobs

Job

Description

ClassifyFetchJob

Runs every minute on the server that is assigned the role of a Classification Server.

It searches the classification/inbox folder for the input files and adds them to the priority queues. One input file can result in multiple snapshots with the name <PRIORITY>_<CRID>_<BATCHID>_<NODEID>_<MSUID>_<TIMESTAMP>_snap<N>.csqlite. The input file contains the location where the actual file has been kept in the classification/content folder. The job also keeps a list of files that could not be fetched.

Note:

Error logs are created in the <Install directory>/log/fetch folder.

ClassifyFetchPauseJob

Runs once every minute on any node that acts as the Classification Server.

Refreshes the pause or resume status of fetch jobs as per the duration configured for content fetching.

CancelClassifyRequestJob

Runs every 20 seconds in Communication Service and Classification Service.

Fetches the list of classification requests that are cancelled and distributes this request between Data Insight nodes.

Before classifying files, all the classification jobs consult this list to identify the requests that are marked for cancellation. If they observe any canceled request in the new request that is submitted for classification, then that request is deleted.

ClassifyJob

Runs once every minute on any node that acts as a Classification Server.

Checks the classification/inbox folder for input files submitted for classification folder and adds them to three separate priority queues. It picks a file from the highest queue in FIFO order, and starts classifying content using Veritas Information Classifier. All files in that input file are submitted for classification. Once all paths in the file have been classified, result of the classification and any resulting errors are written to a database in the classification/outbox folder.

UpdateVICPolicyMapJob

Runs every ten seconds on the Management Server.

It ensures that Data Insight configuration database is in sync with the Classification Policy Manager.

UpdateConfigJob

Reconfigures jobs based on the configuration changes made on the Management Server.

CreateFeaturesJob

Runs once every week on Sunday at 00.01 A.M. on the Indexer.

Checks if sufficient classified data is available for the supervised learning algorithm to create predictions (training sets).

The job has a multi-threaded execution framework which executes actions in parallel. The default thread count is 2. You can set the value using the matrix.classification.sl.features.threads property at global or node level.

Note:

The node level property always takes precedence over the global level property.

PredictJob

Runs once every week on Sunday at 05.00 A.M. on the Indexer.

Copies the prediction files from the temp output directory to a classification outbox.

SLCreateBatchesJob

Runs every 2 hours on the Indexer.

The job creates batches of files for the consumption of Veritas Information Classifier. These files are classified with high priority.

ClassifyManageWorkloadJob

Runs every one minute on the server that is assigned the role of a Classification Server. This job is enabled only on the master Classification Server.

Checks the classification or workload folder on master Classification Server and counts batches based on their priority. If the workload needs to be distributed, the job fetches a list of servers' in it its pool and fetches the number of batches based on their priority in the classification or inbox folder. If the number of batches on any slave that have priority less than 10, then the job distributes the batches across that slave and copies them to the slave's the classification or inbox folder.

See Monitoring Data Insight jobs.