Application Event ID 1000 errors occur hourly on the Compliance Accelerator server.

Article: 100060342
Last Published: 2023-09-07
Ratings: 0 0
Product(s): Enterprise Vault

Problem

The Application Event Log on the Compliance Accelerator (CA) server shows hourly instances of error Event ID 1000 for the .NET Runtime source after upgrading CA to the 14.3.0 or greater release.  This can be associated with the Enterprise Vault (EV) storage server's EV Event Log having warning Event ID 41596 appearing periodically.

 

Error Message

The following errors are from the Application Event Log on the CA server:

Source:        .NET Runtime
Event ID:      1000
Task Category: None
Level:         Error
Keywords:      Classic
Description:
Category: IR.Infrastructure.WindowsEventLogging
EventId: x
SpanId: xxxxxxxxxxxxxxxx
TraceId: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
ParentId: 0000000000000000
RequestId: 0000000x-0000-xx00-x00x-00000x0000xx
RequestPath: /api/learning-enabled-departments/customer/X

Error occurred during metadata refresh.

Exception: 
System.Data.SqlClient.SqlException (0x80131904): Login failed for user 'NT AUTHORITY\ANONYMOUS LOGON'.
   at Symantec.EnterpriseVault.DatabaseAccess.EVSqlBase.Do[T](Func`1 action)
   at Symantec.EnterpriseVault.DatabaseAccess.EVSqlBase.Do(Action action)
   at Symantec.EnterpriseVault.DatabaseAccess.EVSqlConnection.Open()
   at IR.Database.SqlHelper.EVExecuteReader(SQLFunctionArguments sqlArguments)
   at IR.Infrastructure.Security.ImpersonationService.<>c__DisplayClass3_0`2.<RunImpersonated>b__0()
   at System.Security.Principal.WindowsIdentity.<>c__DisplayClass73_0.<RunImpersonatedInternal>b__0(Object )
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext , ContextCallback , Object )
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext , ContextCallback , Object )
   at System.Threading.ExecutionContext.Run(ExecutionContext , ContextCallback , Object )
   at System.Security.Principal.WindowsIdentity.RunImpersonatedInternal(SafeAccessTokenHandle , Action )
   at System.Security.Principal.WindowsIdentity.RunImpersonated(SafeAccessTokenHandle , Action )
   at IR.Infrastructure.Security.ImpersonationService.RunImpersonated[T,T1](Func`2 action, T argument)
   at IR.Infrastructure.Database.MetadataSyncEV.ExecuteImpersonatedQuery(String query, String connectionString, Func`2 action)
   at IR.Infrastructure.Database.MetadataSyncEV.ExecuteQuery(String query, String connectionString, Func`2 action)
   at IR.Infrastructure.Database.MetadataSyncEV.GetAuthorizedPrincipals()
   at IR.Infrastructure.Database.MetadataSyncEV.RefreshAuthorizationCache()
ClientConnectionId:xxx00x00-0000-00xx-xx00-xxx0x00x0x0x
Error Number:18456,State:1,Class:14

V-437-1000


Source:        .NET Runtime
Event ID:      1000
Task Category: None
Level:         Error
Keywords:      Classic
Description:
Category: IR.Infrastructure.WindowsEventLogging
EventId: x
SpanId: xxxxxxxxxxxxxxxx
TraceId: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
ParentId: 0000000000000000
RequestId: 0000000x-0000-xx00-x00x-00000x0000xx
RequestPath: /api/learning-enabled-departments/customer/X

Error occurred while authorizing user :<DomainName>\<UserID>. Error: System.Data.SqlClient.SqlException (0x80131904): Login failed for user 'NT AUTHORITY\ANONYMOUS LOGON'.
   at Symantec.EnterpriseVault.DatabaseAccess.EVSqlBase.Do[T](Func`1 action)
   at Symantec.EnterpriseVault.DatabaseAccess.EVSqlBase.Do(Action action)
   at Symantec.EnterpriseVault.DatabaseAccess.EVSqlConnection.Open()
   at IR.Database.SqlHelper.EVExecuteReader(SQLFunctionArguments sqlArguments)
   at IR.Infrastructure.Security.ImpersonationService.<>c__DisplayClass3_0`2.<RunImpersonated>b__0()
   at System.Security.Principal.WindowsIdentity.<>c__DisplayClass73_0.<RunImpersonatedInternal>b__0(Object )
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext , ContextCallback , Object )
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext , ContextCallback , Object )
   at System.Threading.ExecutionContext.Run(ExecutionContext , ContextCallback , Object )
   at System.Security.Principal.WindowsIdentity.RunImpersonatedInternal(SafeAccessTokenHandle , Action )
   at System.Security.Principal.WindowsIdentity.RunImpersonated(SafeAccessTokenHandle , Action )
   at IR.Infrastructure.Security.ImpersonationService.RunImpersonated[T,T1](Func`2 action, T argument)
   at IR.Infrastructure.Database.MetadataSyncEV.ExecuteImpersonatedQuery(String query, String connectionString, Func`2 action)
   at IR.Infrastructure.Database.MetadataSyncEV.ExecuteQuery(String query, String connectionString, Func`2 action)
   at IR.Infrastructure.Database.MetadataSyncEV.GetAuthorizedPrincipals()
   at IR.Infrastructure.Database.MetadataSyncEV.RefreshAuthorizationCache()
   at IR.Infrastructure.Services.AuthorizationService.IsAuthorized(String userName, AccountType accountType)
ClientConnectionId:xxx00x00-0000-00xx-xx00-xxx0x00x0x0x
Error Number:18456,State:1,Class:14

V-437-1000

 

Cause

The errors are caused by enhancements to the CA Intelligent Review (IR) feature regardless of any IR options selected at the Department or search levels.

This is partially caused by missing Service Principal Name (SPN) data for the SQL Server or SQL Always On Availabilty Group (AOAG) Listener where the CA Customer database resides.

Note:
There is no apparant disruption in CA searching or Random Sampling processing due to these errors.
 

Solution

Two workarounds exist for this issue:
1. Change the interval of the errors from every hour to up to every 5 hours.
2. Suppress the errors from being logged in the Application Event Log.

Workaround 1: Change the interval of the errors from every hour to up to every 5 hours.
1. Log onto the CA Client using an account with a Role assignment having the Modify System Configuration permission, such as the Vault Service Account (VSA).
2. Navigate to the Configuration tab > Settings sub-tab.
3. Expand the Random Capture folder.
4. Scroll down the list of options in the Random Capture folder to locate and select the Stale config timeout (mins) option.
5. Click in the Value column of this option and change the value from the default of 60 to any value up to the maximum of 300.
6. Click anywhere else on the page to change the focus off of the option's Value field.
7. Click the Save button in the lower right of the tab.  Note that there is no restart required for this change to become effective after saving.
8. Monitor the CA server's Application Event Log for new instances of the .NET Runtime Event ID 1000 errors that should occur less frequently than every hour.

Workaround 2: Suppress the errors from being entered into the Application Event Log.
1. Log onto the CA server using an account that has Local Administrator permissions, such as the VSA.
2. Open File Explorer.
3. Expand the CA installation folder, then navigate to Veritas Intelligent Review | IR.APIEndPoint sub-folder (default path is C:\Program Files (x86)\Enterprise Vault Business Accelerator\Veritas Intelligent Review\IR.APIEndPoint).
4. Locate and use a text editor, such as Notepad, to edit the file appsettings.json.
5. Scroll to the bottom of the file.
        Example of the last few lines of the file before adding anything:
            "BackgroundMetadataRefreshIntervalInMinute": "30",
            "LoggingProvider": "Log4Net",
            "AllowWindowsEventLogging": true,
            "ProductType": 2,
            "MaxParallelism": 5
            }

6. Add a comma at the end of the line MaxParallelism flag's line after the value and press the Enter key to insert a new line below this line.
7. Add the flag "SkipImpersonationAndDelegation": true in the new line.
      Example of the last few lines of the file after the adding the comma and the line with the new flag:
            "BackgroundMetadataRefreshIntervalInMinute": "30",
            "LoggingProvider": "Log4Net",
            "AllowWindowsEventLogging": true,
            "ProductType": 2,
            "MaxParallelism": 5,
            "SkipImpersonationAndDelegation": true
            }

8. Save and close the file.
9. Open IIS Manager.
10. Expand the IIS Server to see the Application Pools and Default Web Site folders.
11. Select the Application Pools folder to see in the middle pane, the different application pools that are installed on the server.
12. Select the IRApiEndpoint application pool.
13. Click the Advanced Setttings... link to open the Advanced Settings dialog page.
14. In the Advanced Settings dialog box:
14.1. Locate and select the Identity line.
14.2. Click on the kabob (...) button next to ApplicationPoolIdentity to open the Application Pool Identity dialog page.
14.3. Click the Custom account: radio button.
14.4. Click the Set... button to open the Set Credentials dialog page.
14.5. In the Set Credentials dialog page:
14.5.1. In the User name: field, enter the VSA login in the form of domain\username.
14.5.2. In the Password: field, enter the VSA's password.
14.5.3. In the Confirm password: field, re-enter the VSA's password.
14.5.4. Click the OK button to save the change and return to the Application Pool Identity dialog page.
14.6. Click the OK button to close the Advanced Settings dialog page.
15. Close IIS Manager.
16. Open the Services MMC snap-in.
17. Locate and stop the Enterprise Vault Accelerator Manager Service (EVAMS).
18. Open an administrative Command Prompt.
19. Execute the command iisreset.
20. When IIS has successfully been reset, start the EVAMS.
21. Close the Services MMC snapin.

Important Note: If the VSA's password is changed, the above steps will need to be repeated to change the VSA's password for the IRApiEndpoint application pool's identity.

Upon complettion of these steps, use the Event Viewer to monitor the Application Event Log for any new entries of the .NET Runtime Event ID 1000 errors.  All future instances of the above noted errors should be suppressed and not appear in the event log.

This issue is currently under investigation by Veritas Technologies LLC. Pending the outcome of the investigation, this issue may be resolved by way of a patch or hotfix in current or future revisions of the software. However, this particular issue is not currently scheduled for any release.  If you feel this issue has a direct business impact for you and your continued use of the product, please contact your Veritas Sales representative or the Veritas Sales group to discuss these concerns.
 
Note: Customers experiencing this issue are encouraged to contact Veritas Technical Support as data is still being collected to assist in resolving this issue.

 

References

JIRA : CFT-5397

Was this content helpful?