"Failed to import sensitive file paths from Data Loss Prevention Server java.lang.OutOfMemoryError: Java heap space" is reported by Data Insight

Problem

Data Loss Prevention (DLP) fails to transmit the expected sensitive file data to Data Insight (DI) due to exhausting the memory available to the process thread. The DLP incident report for file objects meeting enable policies does not properly transmit the data to DI.

Error Message

Currently, all versions show this error message with the default noted at 512 MB. The message will be changed to relevant numbers as of version 5.1.2 and 5.2 or newer. For the current defaults please see below. SEVERE: #{1} [ConfigUtils.main] Failed to import sensitive file paths from Data Loss Prevention Serverjava.lang.OutOfMemoryError: Java heap space at java.util.HashMap.createEntry(Unknown Source) at java.util.HashMap.addEntry(Unknown Source) at java.util.HashMap.put(Unknown Source) at com.symc.matrix.xinterfaces.dlp.SensitiveFileExtractor.populateSensitiveFileInfo(SensitiveFileExtractor.java:259) at com.symc.matrix.xinterfaces.dlp.SensitiveFileExtractor.getSensitiveFiles(SensitiveFileExtractor.java:207) at com.symc.matrix.xinterfaces.dlp.SensitiveFileExtractor.getTaggedSensitiveFiles(SensitiveFileExtractor.java:280) at com.symc.matrix.util.ConfigUtils.main(ConfigUtils.java:12174) at com.symc.matrix.report.util.ReportsUtils.main(ReportsUtils.java:6200) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at com.exe4j.runtime.LauncherEngine.launch(Unknown Source) at com.exe4j.runtime.WinLauncher.main(Unknown Source) at com.install4j.runtime.launcher.WinLauncher.main(Unknown Source) #{1} [EventsManager.publishEvent] WARNING: [DLP_SENSITIVE_FILES_IMPORT_FAILURE@<Servername_reference>@matrixuser@matrixdomain] Error importing Sensitive file paths from Data Loss Prevention Server (took: 62457720ms) (Details: OutOfMemoryError detected. SOLUTION: Set higher memory limit for DLP scan by issuing following command on the Management Server [default (mb): 512]: configdb -o -T node -k 1 -J sensitivefilejob.max.memory -j <new_limit_mb>)

Cause

The memory available to the thread has become exhausted from storing the data being transmitted. A java.lang.OutOfMemoryError error that is detected while the DLPSensitiveJob execution is underway will cause the job to fail. 

In DI versions prior to 5.0, the default amount of memory that is allocated to the DLPSensitiveJob execution is 512 MB. Since version 5.0, and newer, this is set to 1024 for 32-bit machines and 2048 for 64-bit machines.

Solution

Run configdb from the Management Server.

C:\Program Files\DataInsight\bin> configdb -o -T node -k 1 -J sensitivefilejob.max.memory -j <new_limit_mb>

Note: In this example, the value 1 corresponds to the node ID of the Management Server. The new_limit_mb is equivalent to the expected necessary memory in in Megabytes (for example, 4 GB is "4096") while remaining under the physical limitations of the available memory on the Management Server.

To identify the current memory settings:

C:\Program Files\DataInsight\bin>configdb -p -T objattr |findstr  sensitivefilejob.max.memory

Example: Once it is changed to 4 GB, it would be seen as the following:

node:1:sensitivefilejob.max.memory=4096

Note: The default memory setting is hard-coded and does not appear in the output of the above print command (-p) until it has been altered from the default. It is safe to assume the defaults are set if there is no result returned by the command. Once altered, the command will show the change.

The default binary files location is:

• For version 5.1, or greater: C:\Program Files\DataInsight\bin
• For version 5.0, or lower:  C:\Program Files\Symantec\DataInsight\bin