Arctera™ Insight Classification Help

This topic provides information on the following:

• Basic components of a condition

• Custom fields

  • Create new property by using custom property fields

  • Create a new property by using YAML configuration file

• Text matches

• Emoji Support

• Variable support

• Regular expression matches

• Author and Recipient department based policy condition

• Pattern matches

• Exact Data Matches

• Language matches

• Entity matches

• Sentiment score

• Risk score and risk level information on classification

• Condition groups

• [[UNRESOLVED XREF: v169816700 ]]

The property specifies the part or characteristic of an item that you want to evaluate: its content, title, modified date, file size, and so on. When you choose a property from the list, the options in the two other fields change to suit it. For example, if you choose the "Modified date" property, the other fields provide options with which you can set one or more dates. For properties such as "Content" the available operators are as follows:

• contains text

• matches regex

• department based policy conditions

• matches pattern

• is similar to

• contains exact data match in

• language is

• contains entity

• sentiment score

Observe the following guidelines when you set up a condition to look for specific words or phrases in the items that you submit for classification:

• The condition can look for multiple words or phrases, if you place each one on a line of its own. An item needs to contain just one word or phrase in the list to meet the condition.

• Select Match Case to find only exact matches for the uppercase and lowercase characters in the specified words or phrases.

• Select String Match to find instances where the specified words or phrases are contained within other ones. For example, if you select this option, the word enter matches enters, entertainment and carpenter. If you clear the option, enter matches only enter.

  Similarly, if you select String Match, the phrase call me matches call media and recall meeting, but not surgically mend.

  Leading and trailing spaces are now considered as part of string match conditions in both policies and patterns. It means if there is a space before first character or after the last character of the policy condition, it will still be considered as part of the evaluation criteria.

  For example, if you want to generate a match for the term so sorry, good hits will be we are so sorry about this or I am so sorry this happened to you but without leading and trailing space there is a possibility to generate false positive like alfonso sorry about that - no biggie!. Inclusion of leading and trailing spaces will help in avoiding such false positives and you can make use of a space character before and after to classify only so sorry.

• You can place the proximity operators NEAR and BEFORE between two words in the same line. For example, tax NEAR/10 reform matches instances where there are no more than ten words between tax and reform. sales BEFORE/5 report matches instances where sales precedes report and there are no more than five words between them. The number is mandatory in both cases.

  Note:

  These proximity operators may not work as expected when evaluating formatted data, such as tables and spreadsheets. The conversion process that this data undergoes before it is classified can swap the order of the table cells. For example, suppose that a spreadsheet contains the word sales in one cell and report in the cell immediately to the right. This should match the operator sales BEFORE/5 report but may not do so after the spreadsheet has been converted, because the conversion process has transposed the two words.

• Word and phrases can include the asterisk (*) and question mark (?) wildcard characters. As part of a word, an asterisk matches zero or more characters. On its own, the asterisk matches exactly one word. A question mark matches exactly one character. For example:

  • stock* matches stock, stocks, and stockings.

  • *ock matches stock and clock.

  • *ock* matches stock and clocks.

  • ??ock matches stock and clock, but not dock.

  • sell * stock matches sell the stock and sell some stock, but not sell stock.

  You can use wildcards in combination with the NEAR and BEFORE operators. For example:

  • s?l? BEFORE/1 stock* matches sold the stock, sell stocks, and sale of stockings.

• Select Exclude Match if you want to exclude specific words or phrases while evaluating the policy condition criteria.

  When you select this option, along with the inclusion terms, you can also define the terms that you want to exclude from the matching criteria.

  For example, assume that a document contains a sample text "Admin: There is a spoofing activity detected. Bob: Can you help me locate a spoofed account for spoofed email account?". You want to hit on the terms "spoof, spoofed, spoofing" only, and want to avoid or exclude the terms spoofed email account, an email spoof. In such a scenario, you can provide the keywords "spoof, spoofed, spoofing" in the inclusion terms field, and provide the terms "spoofed email account, an email spoof" in the exclusion terms field as shown in the sample image below.

  Keyword explorer

  In case the list of keyword is long, you can manage it by clicking Expand at the bottom of the list.

  A new pop-up with list of all keywords appears on the screen.

  You can search and sort this list by using the search box or clicking Sort. You can also export the list in CSV format by clicking Export.

  If you want to add or import list of keywords, click Edit.

  • For built-in policy, click Edit to

    • enable/disable status

    • change tags, risk weight, confidence levels

    • adjust minimum match counts

  • For custom policy, click Edit to

    • enable/disable status

    • change policy name and description, tag, risk, and weight

    • update policy conditions

  • Click Copy to create a fully editable version of the existing built-in or custom policy.

  In the right pane, click Edit Conditions.

  Click Expand in the Conditions section.

  On the Keyword Condition pop-up, you can perform following actions.

  • Add new keyword

  • Search a keyword in the existing list.

  • Import keywords in CSV format.

  • Export keyword list in CSV format.

  Note:

  Keyword Condition can contain maximum 11,000 keywords. For example, if you already have 2000 keywords in the condition, you can add 9,000 more keywords only.

  Using Special Standalone Characters in Keyword-Based Policies

  When creating keyword-based conditions in policies, it is important to understand how the application handles special standalone characters like @, #, or $.

  • Special standalone characters are not allowed by default in keyword conditions.

  • If such characters are entered, the system will display a warning message.

  • The policy can still be saved, but any standalone special characters will be automatically removed in the back-end, unless the String Match option is explicitly selected.

  To Retain Special Characters

  If you want to retain standalone special characters in your keyword conditions, you must select the String Match option. This setting ensures that the characters are preserved exactly as entered and bypasses the automatic cleanup process.

  Scenarios and Behavior

  Scenarios	Behavior
  Policy with keyword condition containing special characters	Warning message is shown. Special characters will be removed unless String Match is selected.
  Policy with keyword condition and String Match selected	Warning message is not shown. Special characters are retained.

  Limitations of the exclusion policy condition

  • This field allows only keywords based exclusion. It means the input field for exclusion terms will only accept keywords, and not regular expression, pattern, and so on.

  • Keyword based exclusion works only for the scenario where every inclusion term is completely contained within an exclusion term.

  • The group-level condition proximity option will not be available for a group if any of the underlying conditions has exclusions.

  To use the exclusion policy condition, See Using a keywords-based exclusion policy condition.

• A variable can have up to 100 newline-separated values.

• In a text-type condition, a single phrase can include up to two variables, including duplicates.

• In a policy's text-based condition, if variables are used, the exclude match feature cannot be applied.

• Variables are not permitted in the exclude match text box.

• If a variable is misspelled, not defined, or not used with the correct syntax (for example, {{variable_name}}), it is treated as plain text during classification and matched accordingly.

• After removing multiple items from pinned department panel rapidly and closing it, reopening that panel might take a while.

Note the following if you do not get the expected results when you test a policy that makes use of a built-in pattern:

• It is important to check that your test item meets the pattern confidence levels. For example, by default, the Credit Card Policy looks for content that matches the pattern "Credit/Debit Card Number" with medium to very high confidence. To meet the requirements of the medium confidence level, an item must contain either of the following:

  • A delimited credit card number (one that contains spaces or dashes between the numbers).

  • Both a non-delimited credit card number and one or more credit card keywords, such as "AMEX" or "Visa".

  So, an item does not meet these requirements if it contains a non-delimited credit card number but it does not also contain credit card keywords.

• After you click Show details to view the results of a test, the Test classification results window may fail to highlight some or all of the matches. This is a known issue with certain patterns only. A future version of the Arctera Insight Classification will correct the issue.

To classify information using Exact Data Match

• Create an EDM pattern by setting the configuration options and providing the source document (typically containing the desired fields exported from a data store, such as a database). See “To create an Exact Data Match based pattern”.

• Use the resulting EDM pattern in any policy to be used for EDM based classification.

Exact Data Match provides following benefits:

• Provides the ability to detect specific data sets from a database. For example, employee records.

• Supports matching of combinations of data. For example, matching one or more fields and optional fields as per configured proximity value.

• Supports large data sets like database records.

• Provides data protection by hashing of stored data.

• Automatically synchronizes the exact data match rule pack (which is required for classification) on the remote classification servers. Manual intervention is not required.

• Supports file encryption for the exact data match rule pack files with the mechanism similar to tenant-specific patterns, policies, and tags.

• Supports the Min/Max disposition for exact data match type policy conditions while configuring policies.

• Supports text in all languages.

Classification performance for Exact Data Match based policy depends on following factors.

• Number of records to be matched

• Number of fields and field size

• Data being classified

• Number of matches

• Proximity and column matches found

• Compute hardware and available resources

• Apache Tika Engine: The default option, which has comparatively lower speed and accuracy than FastText.

• FastText Engine: A newly introduced engine that offers faster and more accurate detection for both single-language and multilingual text.

• score = 0 or score < 50 is negative

• score = 50 is neutral

• score = 100 or score > 50 is positive

• is - Sentiment analysis is performed, and the policy tag will be returned only if the sentiment score matches this number

• is at most - Sentiment analysis is performed, and the policy tag will be returned only if the sentiment score is less than or equals this number

• is at least - Sentiment analysis is performed, and the policy tag will be returned only if the sentiment score is equal to or higher than this number

• is in range - Sentiment analysis is performed, and the policy tag will be returned only if the sentiment score falls in this range

• Export is not allowed with format type policies.

• If multiple policies are selected for export and any policy is of format type, then all policies except the format type policy will be exported, depending on their types.

• After a format type policy is created, if the feature is disabled, classification for format type policies will not be allowed.

• lowRiskUpperLimit - It can be zero or greater than zero. By default, it is set to 2.

• mediumRiskUpperLimit -It can be any non-zero positive integer. But must be greater than lowRiskUpperLimit value. By default, it is set to 5.

• highRiskUpperLimit -It can be any non-zero positive integer. But must be greater than mediumRiskUpperLimit value. By default, it is set to 10.

• The matchDetailLevel is configured in classify request as either LOW/MEDIUM/HIGH

• The item must have some risk based on the risk score and the risk level limits settings in the YAML file.

• Sentiment score/Named Entity based policy condition hits does not contribute towards risk score.

• Contribution to the total item risk score will be zero due to Subscription policy and any language detection policy as all these policies have risk weight zero by default.

• Due to language detection policy hits, some discrepancies may be observed in Most Common Sensitive Data results from analyzer overview page. The result of the analyzer overview page is accurate.

• Following policy conditions contribute to risk score.

  • Content

  • Title

  • Author

  • Content Type

  • Recipient

  • Modified Date

  • Creation Date

  • Sensitivity

  • Category

  • Size (Bytes)

  • Custom date field

  • Custom number field

  • Custom string field

  • Sender and Recipient department based policy condition

You can group a set of conditions and nest grouped conditions within other grouped conditions. The group operator that you choose determines whether an item must meet all, some, or none of the conditions in the group to be considered a match. The following group operators are available:

• All of. An item must meet all the specified conditions.

• Any of. An item must meet at least one of the specified conditions.

• None of. An item must not meet any of the specified conditions.

  Note:

  You can nest a None of group within an All of group to look for certain condition matches while also excluding others. For example, to achieve the effect of "(condition X AND condition Y) BUT NOT condition Z", you would include the X and Y conditions in an All of group and the Z condition in a nested None of group.

• n or more of. An item must meet the specified number of conditions.