Expired backup data does not get removed from PureDisk Server

Problem

PureDisk clients backing up to the SPA for an extended period of time.
Disk space on /Storage has reached its high water mark.
PDDO is Out of space
PureDisk cleanup not working
Cannot reclaim disk space in /Storage
Storage is full on PureDisk 6.6 server

Cause

Policy configuration issues (PureDisk cleanup policies not running).

Backing up more data than the current storage space can accomodate or poor deduplication rates on backups

PureDisk binaries not up to date with the latest fixes relating to storage and cleanup.

Solution

If the available space on a PureDisk 6.5 or PureDisk 6.6 server reaches its highwater mark, backups will stop functioning.  There are many reasons why space allocated to the /Storage/data directory will fill up.

Below is a list of the most common reasons.

The amount of storage is not adequate for the amount of data being backed up by the clients.  This is determined by the answers to the following questions.

• How much data is being backed up on a client?
• What type of data is being backed up by the client?
• How many clients are being backed up to this PureDisk server?
• What is the deduplication rate for the data on each client?
• What is the retention level for the data being backed up and is it adequate?

See article https://isearch.veritas.com/internal-search/en_US/article.100004389.html Methods to determine how much space backups are taking on a deduplication storage unit MSDP or PDDO using NetBackup 7.01

The Puredisk cleanup policies are not running successfully, or not running often enough.

1. Is there a data removal policy running regularly for each data selection type?  For example, how often do the PDDO Data Removal, data selection removal, CR Garbage Collection, MB Garbage collection and CR Queue processing policies run?  Run each one at a time and watch to see if space is being reclaimed.  Perhaps they are not running often enough.
2. Are there any client backups being done using redundant data selections?  If a client is being backed up using two data selections that list data that does not deduplicate well, that will take up twice as much space for the same data.  For example is there one data selection backing up System State (which does not duplicate very well) and another one backing up shadow copy components on the same client? If yes, then delete one of those data selections and run the cleanup steps.

The PureDisk binaries are not patched with the latest cleanup fixes.

If space is still not being reclaimed, gather the following in depth command and report outputs needed for an escalation.  Gather these things from the PureDisk SPA.

Run the Datamining workflow using these steps.

Click the Manage-Policies.

• On the left, expand Storage Pool Management Policies
• click the plus sign (+) to the left of Data Mining. 
• click the System policy for data mining.
• Enable the policy. Click Edit policy in the left pane to verify that the policy is enabled. If it is disabled, enable it.

In the left pane, click Run Policy.

Gather the Data mining report.

Click the Reports tab, then Data mining report.

• Select the scope of the data mining report.
• Choose the level:  A storage pool
• Click Show Data Mining Report in the left pane.
• Either get a screen shot of this or select the data in the report and copy it to a text document.

Gather the available disk space statistics by running this command at the root command prompt.

Gather /Storage file system information:
 

Run this command as root to list the tables in the metabase:

Run this command as root to get a list of dataselections and sizes:

Copy and paste the command exactly as listed and then redirect output into a text file.

Use the above outputs to verify that the client backup data statistics have been examined and understood and that the manual data removal steps from the "How to manually cleanup disk space" article have been followed as documented.
 
Gather the pdgetlogs output from the SPA.

Run the PDgetlogs.sh script and send in the output for review.  The script collects all of the support logs and puts them into a tar file for sending to Technical Services.

The script is located in the following location:
 

The output file is:  /tmp/PDgetlogs_

Applies To

PureDisk 6.6 all in one SPA

Backing up clients using the Puredisk agent

Backing up clients using NetBackup via PDDO plugin