While searching on a large quantity of Index Volumes, the performance is extremely slow causing the User Interface to become unresponsive

Article: 100005724
Last Published: 2015-05-06
Ratings: 0 0
Product(s): Enterprise Vault

Problem

When the Accelerator product is performing a search on a very large quantity (typically in the tens of thousands) of Index Volumes, the Accelerator Service process consumes virtually all available memory on the Accelerator Server. The limited memory resources cause the User Interface to become unresponsive as well as volume processing to become extremely slow; in some cases, taking up to days to complete the search.

 

Error Message

Event Type: Error
Event Source: Accelerator Service Processor
Event ID: 93
Description:
APP AT - Customer ID: X - An error occurred while updating Vaults for Vault Store: ID=X KVS ID: 1234567890ABCDEF1234567890ABCDEF12345678test.com. System.OutOfMemoryException: The EnterpriseVault.DirectoryConnection object reported an error.
Ran out of memory

Event Type: Error
Event Source: Accelerator Service Processor
Event ID: 117
Description:
APP AT - Customer ID: X - Error adding Background SQL to the queue. System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
   at System.String.Concat(String str0, String str1, String str2)
   at Microsoft.Win32.RegistryKey.OpenSubKey(String name, Boolean writable)
   at System.Data.Common.ADP.LocalMachineRegistryValue(String subkey, String queryvalue)
   at System.Data.SqlClient.SqlConnectionString..ctor(String connectionString)
   at System.Data.SqlClient.SqlConnectionFactory.CreateConnectionOptions(String connectionString, DbConnectionOptions previous)
   at System.Data.ProviderBase.DbConnectionFactory.GetConnectionPoolGroup(String connectionString, DbConnectionPoolGroupOptions poolOptions, DbConnectionOptions& userConnectionOptions)
   at System.Data.SqlClient.SqlConnection.ConnectionString_Set(String value)
   at System.Data.SqlClient.SqlConnection.set_ConnectionString(String value)
   at KVS.Accelerator.Common.BackgroundSQL.InitializeComponent()
   at KVS.Accelerator.Common.BackgroundSQL..ctor()
   at KVS.Accelerator.Common.BackgroundSQLTaskQueue.QueueWork(ThreadSafeQueue& theQueue, Object myData).

Event Type: Error
Event Source: Accelerator Service Processor
Event ID: 137
Description:
APP AT - Customer ID: X - An error occured in ProcessingMultiQueues::QueueingThread. System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
   at System.Collections.Hashtable.ValueCollection.GetEnumerator()
   at KVS.Accelerator.Common.ProcessingMultiQueues.SignalQueuingFinishedEvent(Boolean Set)
   at KVS.Accelerator.Common.ProcessingMultiQueues.QueueingThread().

Event Type: Error
Event Source: Accelerator Service Processor
Event ID: 223
Description:
APP AT - Customer ID: X - An error occurred. System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
   at System.Data.SqlClient.TdsParser.ProcessEnvChange(Int32 tokenLength, TdsParserStateObject stateObj)
   at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
   at System.Data.SqlClient.SqlDataReader.ConsumeMetaData()
   at System.Data.SqlClient.SqlDataReader.get_MetaData()
   at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
   at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, DbAsyncResult result)
   at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method)
   at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior, String method)
   at System.Data.SqlClient.SqlCommand.ExecuteReader()
   at KVS.Accelerator.Production.ProductionQueue.DoCheckForNewProductionRun().

Event Type: Error
Event Source: Accelerator Service Processor
Event ID: 293
Description:
APP AT - Customer ID: X - Error adding Vault to Search in the queue. System.Data.SqlClient.SqlException: The query processor could not start the necessary thread resources for parallel query execution.
   at KVS.Accelerator.Search.SearchVaults.GetVaultToSearchDS(Int32 SearchID, VaultSearchStatus Status, Int32 IndexServerID)
   at KVS.Accelerator.Search.SearchVaults.GetVaultToSearchToAddToQueueDS(Int32 IndexServerID)
   at KVS.Accelerator.Search.SearchQueues.SearchVaultQueueWork(ProcessingMultiQueues TheProcessingMultiQueues, Object myData).

Event Type: Error
Event Source: Accelerator Service Processor
Event ID: 323
Description:
APP AT - Customer ID: X - An error has occured when initializing the Search Vaults. System.Data.SqlClient.SqlException: The query processor could not start the necessary thread resources for parallel query execution.
   at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)
   at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection)
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj)
   at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
   at System.Data.SqlClient.SqlDataReader.HasMoreRows()
   at System.Data.SqlClient.SqlDataReader.ReadInternal(Boolean setTimeout)
   at System.Data.SqlClient.SqlDataReader.Read()
   at System.Data.Common.DataAdapter.FillLoadDataRow(SchemaMapping mapping)
   at System.Data.Common.DataAdapter.FillFromReader(DataSet dataset, DataTable datatable, String srcTable, DataReaderContainer dataReader, Int32 startRecord, Int32 maxRecords, DataColumn parentChapterColumn, Object parentChapterValue)
   at System.Data.Common.DataAdapter.Fill(DataTable[] dataTables, IDataReader dataReader, Int32 startRecord, Int32 maxRecords)
   at System.Data.Common.DbDataAdapter.FillInternal(DataSet dataset, DataTable[] datatables, Int32 startRecord, Int32 maxRecords, String srcTable, IDbCommand command, CommandBehavior behavior)
   at System.Data.Common.DbDataAdapter.Fill(DataTable[] dataTables, Int32 startRecord, Int32 maxRecords, IDbCommand command, CommandBehavior behavior)
   at System.Data.Common.DbDataAdapter.Fill(DataTable dataTable)
   at KVS.Accelerator.Search.SearchVaults.GetVaultToSearchDS(Int32 SearchID, VaultSearchStatus Status, Int32 IndexServerID).

Event Type: Error
Event Source: Enterprise Vault
Event ID: 40966
Description:
A program fault has raised an exception.
Exception: Exception of type 'System.OutOfMemoryException' was thrown.
Diagnostic:
Type: System.OutOfMemoryException
Reference:
Command Line: ".\AcceleratorService.exe" -URL:tcp://10.10.10.10:9501/81e60b38_5608_4cac_aede_e1bf0ac983b1/KVS.EnterpriseVault.ProcessManager12036987 -URL:ipc://EVAccelerator:AcceleratorManager/81e60b38_5608_4cac_aede_e1bf0ac983b1/KVS.EnterpriseVault.ProcessManager12036987 -CUSTOM:2|2|4
Application Domain: AcceleratorService.exe
Process Id: 4800
Thread Id: 9400
Stack Trace:    at System.Collections.Hashtable.HashtableEnumerator.get_Current()
   at System.Runtime.Remoting.Channels.SocketCache.TimeoutSockets(Object state, Boolean wasSignalled)
   at System.Threading._ThreadPoolWaitOrTimerCallback.PerformWaitOrTimerCallback(Object state, Boolean timedOut).

 

 

Solution

This issue has been addressed as part of the following release:


Hotfix for Veritas Enterprise Vault (EV) Discovery Accelerator (DA) 10.0, Build 10.0.0.1159
https://www.veritas.com/docs/000085632

Hotfix for Veritas Enterprise Vault (EV) Discovery Accelerator (DA) 10.0.1, Build 10.0.1.1069
https://www.veritas.com/docs/000085635

 

 

References

UMI : V-437-40966 UMI : V-437-93 Etrack : 293 Etrack : 40966 UMI : V-437-137 UMI : V-437-323 UMI : V-437-117 Etrack : 223 Etrack : 323 Etrack : 137 Etrack : 117 UMI : V-437-293 Etrack : 93 Etrack : 2273560 UMI : V-437-223 Etrack : 2273564

Was this content helpful?