At times, it is required to understand and identify the Retention value information in order to understand the expiry process. This article represents how Retention Category values for Archived items from Enterprise Vault (EV) are applied, identified and enforced in a Centera Storage Device using EMC Governance or Compliance Edition Plus Models.
Since EMC Centera is a Compliancy Storage Device, dependant on the Model type, any item written to the device cannot be modified after initially storing the data.
There are 3 General Model types for EMC Centera:
- Basic Edition
- Governance Edition (GE)
- Compliance Edition Plus (CE+)
Identification of Model Type:
Enable Enterprise Vault's Dtrace on the StorageOnlineOpns* process and restart the Enterprise Vault Storage Service. (See note below)
CVaultStoreEMCCentera::SetPool -- Compliance Device mode:
- Compliance Device mode: True
The Centera Storage is a Governance Edition or Compliance Edition Plus model
- Compliance Device mode: False
The Centera Storage is a Basic Edition model
The primary difference between the Basic Edition and Governance/Compliance Editions are the following:
- Basic Edition leverages the Application based Retention periods exclusively to control deletion requests.
- Compliance/Governance Edition has Storage based Retention as well as Application based Retention to control deletion requests.
Details on deletion requests performed against Centera:
- Enterprise Vault will not perform Privileged Delete requests.
- If not leveraging Collections, delete requests are performed against the EMC Centera Replica Nodes and then Primary Nodes individually when a delete request is performed.
- A successful delete response must be received via Centera from both the Replica and Primary Nodes for EV to identify a delete as successful.
- With EV Centera leveraging Collections, a delete of a Clip and the items associated with the Clip are not performed until all items are marked as deleted which are referenced within the Clip.
Storage and SQL
For an EMC Centera device, a Centera Device has a proprietary storage method. Files are 'presented' to the Storage API (Provided by EMC) by the Application (Enterprise Vault) and the Storage API communicates to the Storage Device. When the items are stored on a Centera Device, these objects are referenced as 'C-Clips' for Content Addressing on the device. A C-Clip, or commonly referenced as 'Clip', is assigned to the data of the object stored. Centera separates the data object, into individual Binary Large Objects (BLOBs) for Single Instance Storage, which are individually referenced in the Clip. A request to retrieve or access an archived item by the application will include the application item reference (Saveset ID) and the Centera Clip, which is passed through the Storage API. The Storage API will leverage the Clip identifier to located the data content requested. A 'BLOB read' request is then performed to collect the BLOB data assigned to the Clip and File allocation. This BLOB data is then converted and presented to the application in its native format.
For additional information how Centera Stores data using the Centera SDK see the following articles:
a. This process is the same with EV 2007 and earlier as well as EV 8.0 and above.
b. Items are placed into the 'staging' area by the StorageArchive process, the StorageFileWatch process will present these items to the EMC Storage API.
c. DVS Files are placed in the 'staging' area utilizing the entire Saveset identity, which includes the unique IdTransaction.
When items are stored in EV, each email is assigned a 'Saveset' identity. See the following article for details on how a Saveset is referenced within the Enterprise Vault Vault Store database:
When items are 'Collected', the Collection reference is stored in the EnterpriseVault Vault Store Database (DB) in the Collection Table as the RelativeFileName value. When Collections are not enabled for Centera, the Clip ID reference is stored in the SavesetStore Table as the StorageIdentifier value. See the following article on identifying the Clip Id associated with a Saveset ID:
Each Saveset is assigned a Retention Category when stored. This can be observed in the Saveset Table as the RetentionCategoryIdentity value. This value refers to the RetentionCategoryEntry Table within the EnterpriseVaultDirectory DB (Ex. RetentionPeriodUnits of "3" equals "Years"), and directly matches the EnterpriseVaultDirectory.RetentionCategoryEntry.RetentionCategoryIdentity value.
("<VaultStoreDBName>" = The Vault Store Database name - do not include the < >)
The following SQL query will return a count list of all the Savesets, grouped by their Retention Category value:
INNER JOIN EnterpriseVaultDirectory.dbo.RetentionCategoryEntry
ON Saveset.RetentionCategoryIdentity = EnterpriseVaultDirectory.dbo.RetentionCategoryEntry.RetentionCategoryIdentity
GROUP BY EnterpriseVaultDirectory.dbo.RetentionCategoryEntry.RetentionCategoryIdentity, EnterpriseVaultDirectory.dbo.RetentionCategoryEntry.RetentionCategoryName
ORDER BY EnterpriseVaultDirectory.dbo.RetentionCategoryEntry.RetentionCategoryIdentity, EnterpriseVaultDirectory.dbo.RetentionCategoryEntry.RetentionCategoryName
There are 2 primary rules regarding Retention categories:
B. If an existing entry in the RetentioncategoryEntry Table is modified (RetentionPeriod and RetentionPeriodUnits), or the existing Retention Category is modified within the Vault Admin Console, all Savesets assigned to that RetentionCategoryIdentity will be affected.
When Centera Stores items referenced to C-Clips, it also generates a 'Metabase' file for the Clip. This descriptor file references the 'retention.period' value and the SavesetId value(s) associated with the Clip. The CDF file can be 'requested' from the Storage Vender (EMC). Normally, in order to request the CDF, the Clip ID must be known and the file must be saved as XML. This file gives an overall view of how the Clip was stored. See the following articles for details on attaining the CDF from Centera for an archived item:
The "retention.period" value is assigned to the Clip at the time of archival and cannot be modified after the item has been stored. The value is in seconds so a proper conversion is required (Ex. the Value of "80438400" would need to be divided by 60 to convert it to minutes, then divided again by 60 to convert it to hours, then divided by 24 to convert it to days then divided by @365 to get the approximate number of years which would be about 2.5 years).
d. If the Retention Category has "Retain items forever" selected, the Clip "retention.period" will be "0", since 'forever' is not a numerical value.
a. Select an item and view it within the context of Search, highlight the Internet Explorer (IE) Address and copy the address into a text editor.
b. From the content of the address, locate the value "SavesetId=" and copy only that value to a different line to separate it from the address.
2. Once the SavesetId value is identified, either via Shortcut or Indexable content, continue through Article TECH51689 and perform the following SQL query against the Vault Store DB:
4. Utilize the following SQL Query to Identify the RetentionCategoryName.
SELECT RetentionCategoryIdentity, RetentionCategoryName
WHERE RetentionCategoryIdentity = 'VALUE'
5. Open the Vault Admin Console and locate the Retention Category name to identify the Retention Value.
- Factors known:
- Item has archive date of January 2005.
- The Retention is set in EV for 5 Years.
- Expiry is set to Archive Date.
- Current Date : January 2010
Based on the above values, when Storage Expiry runs, EV will attempt to delete this item, however the items fail to be deleted.
Why was the item not expired?:
1. Confirm if the Vault Store in question utilizes Collections
a. Open the Vault Admin Console (VAC)
b. Expand Vault Store Groups and expand the associated Vault Store Group
c. Select the Vault Store.
d. Right-click on the Partition value on right side and select Properties.
e. Select the Collections Tab.
2. Utilize Technical Article TECH43224 to locate the Clip Id associated with the Saveset.
3. Retrieve the CDF.xml for the Clip and Identify the retention.period utilizing HOWTO97017 (EVSVR) or TECH65517 (JCASScript)
4. In this example, the converted value of the retention.period in the CDF is 7 years.
The above scenario was referencing a Journal Archive that currently showed as being associated with a 5 Year Retention however items would not expire. The items in question were archived initially using a 7 year retention period, identified in the CDF. After this point, it appeared that the retention period for the Journal Archive was changed from using a 7 year period to a 5 year period.
This change is solely in SQL and did not alter items already stored using the initial 7 year category.
Since the items were initially archived under a 7 year retention, Centera also assigned a 7 year retention to the physical Clip on Storage. These items would not expire until the original 7 year retention has expired.