A new wizard has been added to the indexing tools which will allow validation of an index to take place.
The following options are available when creating a new Verify task:
A full check of the index volumes and its content. If desired a complete verify can also:
a. Provide additional information in the sub task report for items that have missing content.
b. Automatically create a synchronize sub task for index volumes that are found to be out of date (i.e. they have pending additions/deletions)
A quick check that an index volume is correct. This check takes a few seconds to run for each index volume.
A Verify job will be created for each index volume and displayed in the Vault Admin Console (VAC), if the user chooses more than one archive, they will be grouped under a single parent node so that progress can be easily monitored. While an Index volume is being verified, indexing operations are blocked, however the index is still available for searching.
The Verify Process of an Index Volume:
1. Checks that the sub task is running on the correct computer.
In some cases a sub task can be created against a volume that has yet to be allocated a root path. If the sub task runs after it’s been allocated a root path and the root path is on a different server, this check blocks & fails the sub task and reports an error. The admin should delete the sub task and re-create in this case.
2. Checks for the failed volume reason.
A failed reason of metadata sync error (17) indicates there was a problem with the Indexing service startup sync of the volumes. This volume’s failed reason will be cleared after the next successful sync on Indexing service startup. The verify sub task will fail and report that there was an error with accessing the indexing engine metadata.
3. Checks for Deferred Indexing.
If the archive is set to deferred indexing, the sub task will report that deferred indexing is on and will complete with success.
4. Checks for empty volumes with pending work in the offline queue.
If the volume is 64-bit, has no items indexed (no HighestItemSequenceNumber) but has some items in the offline queue, then the verify sub task will attempt to load the collection to allow the indexing engine to process this work. This step was added to work around an issue where the indexing engine would sometimes fail to pick up the volume and process the offline queue even when quiet.
5. Checks for empty volume.
If the volume has no items indexed (no HighestItemSequenceNumber) then the sub task completes without error reporting that there is nothing to check.
6. Checks for offline volume.
If the volume is found to be offline, then the sub task will be set to "Will Retry" and will be retried again in 30 minutes. This should only happen if the volume was set offline in the time between the sub task being picked up for processing and starting processing. If offline originally, the sub task would be skipped at the database level.
7. Set volume to read-only.
This is performed to block additions, updates and deletions on the volume whilst verifying content.
8. Checks the physical location of the volume to ensure it exists.
If not found an error will be reported and the sub task will fail. In addition, the data set statistics are checked for 32-bit (AV) volumes. If incorrect, the sub task will be failed.
9. Checks that collection exists. For 64-bit volumes, this will ensure that the collection exists.
If not found, the verify sub task will create it (pointing at existing location in previous step). If pointing at the wrong location, then the collection will be modified to point at the new location.
10. Checks state.dat file.
For 64-bit volumes Enterprise Vault attempts to load the state.dat file from the index volume and check read properties
11. Checks that the highest sequence number in the index is not higher than the highest sequence number in storage
This indicates that the storage database has been restored and the index is ahead. This is a hard stop error requiring the user to run the Enterprise Vault Storage and Verification Repair (EVSVR) tool to correct the problem. This will be reported to the user.
12. Reports if the volume contains mixed schema
This information comes from the database and not by looking at the volume.
13. The Complete option performs item level verification of the volume.
- Checks for missing docs (Iterates through the index looking for "holes" in the index to find missing docs, then compares this list against EV to confirm if the docs are missing.
- Updates the FailedItemsCount if it finds any in the Index Volume and also updates the ItemAdditionStatusLog table for 64-bit index volumes or the missingitems.log file for Alta Vista (AV) index volumes (32-bit)
- For 64-bit volumes, check for extra documents in the index. This could be due to an item being expired/deleted from the archive but the item remaining in the index (Due to restore of Index volume being out of synch)
- Updates a new ExtraItems count field in the Index Volume with the number of extra documents
- Adds the item to a the ItemDeletionStatusLog table in the Vault Store database that can be used by the synchronize tool
- Searching for extra documents is not supported for 32 bit volumes.
- Report for missing content (E.g. Due to Encrypted content that cannot be decrypted), if the user selected this option when the task was created. This will be performed by searching for documents within the index with the COMR property set in the index. This indicates the reason for the content being missing. When items found:
- For 64-bit indexes, updates the ItemAdditionStatusLog table to include the items and the reason
- For 32-bit (AV), these items are still listed in the IndexMissing.log file. This will be updated if required to reflect the actual items.
14. Clears failed volume reason.
If the verify sub task has reached this point, then it’s assumed the volume is stable and the failed reason is cleared if set
15. The volume is set to read-write state.
At this stage the index volume is available for the processing of indexing work (additions, updates, deletes).
16. If running a full verification against a 64-bit volume, then a search of the volume is attempted.
This is to work around a similar issue to the offline items (See previous reference above).
17. Create synchronize sub task.
If a complete verify was selected and some missing or extra items have been found, a synchronize sub task will be automatically created to attempt the repair. This will only be performed if the option was chosen in the wizard.