Search <book_title>...

NetBackup™ Deduplication Guide

Last Published: 2022-03-28

Product(s): NetBackup (10.0)

Tuning options

Tuning the Graceful mode

To have a faster Graceful mode, one can leverage the CheckSysLoad, BatchSize, and SleepSeconds options to tune the behavior and performance of the Graceful mode.

With a larger number for BatchSize and a smaller number for SleepSeconds, the Graceful mode runs more continuously. When you turn off CheckSysLoad, the Graceful mode keeps running while backup, restore, duplication, replication, compaction, or CRQP jobs are active. Such changes can make the Graceful mode more aggressive, although not as aggressive as the Aggressive mode. The advantage is the tuned Graceful mode has less effect on the system performance than the Aggressive mode for backup, restore, duplication, and replication jobs. It has even less effect than the Aggressive mode with the lowest level 2. The trade-off, especially when CheckSysLoad is turned off, is that it becomes semi-aggressive. It can affect the system performance for the active jobs and it makes the CRC check, CRQP processing, or compaction take a longer time to run and finish.

Tuning the Aggressive mode

Aggressive mode has three levels, 2-4. The higher level means more aggressive and usually better performance for the Encryption Crawler. It also means more effect on the system performance for backup, restore, duplication, replication jobs.

For the best performance of the Encryption Crawler, use Level 2-4 for the Aggressive mode based on the daily system loads. Otherwise, use Level 1 for the Graceful mode. Please note that the Aggressive mode with a higher level doesn't result in a better overall system performance for both the Encryption Crawler and the active jobs. It doesn't mean that the Aggressive mode performs better than the Graceful mode either. You may need to monitor the progress of the Encryption Crawler and the system effect for the active jobs to find the best fit.

You can consider dynamically switching between the Aggressive mode and the Graceful mode for a period of a half day to multiple days. Make the changes according to the pattern of the daily system loads and active jobs. Dynamically switching helps you to discover which mode works for your environment.

Turn on Encryption Crawler for part of the MSDP partitions to reduce system effect

The Aggressive mode affects the performance of backup, restore, duplication, and replication jobs. The tuned Graceful mode does as well, although not as seriously as the Aggressive mode. To reduce the system effect, one can selectively have the Encryption Crawler turned on for part of the MSDP partitions at the same time.

Selectively disable DataStore Write for the MSDP partitions to reduce system effect

The Aggressive mode affects the performance of backup, restore, duplication, and replication jobs. The tuned Graceful mode does as well, although not as seriously as the Aggressive mode. To reduce the system effect, you can selectively disable DataStore Write for the MSDP partitions which have Encryption Crawler running. It can be done with the crcontrol --dswriteoff command for BYO setup. For a NetBackup appliance, the command should be executed through the CLISH. Otherwise the NetBackup appliance resets the state automatically after a short time.

You must reset the DataStore Write state when the process finishes to allow the partitions to take in new backup data.

Tuning recommendations for the Encryption Crawler

Table: Tuning recommendations

Actions	Explanation
Turn on Encryption Crawler in the Graceful mode with the default settings.	Veritas recommends that you wait for fingerprint cache loading to complete before you perform any backups or turn on the Encryption Crawler. Determine when to start by monitoring the `spoold` log and waiting for the message that begins with ThreadMain: Data Store nodes have completed cache loading. The Encryption Crawler is in the Graceful mode by default when you start it. After you start Encryption Crawler, allow it to run for 24 hours to 48 hours with normal backup, duplication, and replication jobs. After this time, the progress of Encryption Crawler process can be checked with the crcontrol --encconvertstate command. After you check on the Encryption Crawler process, review the following: First, check the Progress item and confirm Encryption Crawler progress. If there is no progress or not in the expected speed, you need to make changes to make faster process. Use the Progress item to extrapolate how long Encryption Crawler is expected to take. For example, if 3.3% of the pool is completed in 24 hours, the process may take about 30 days to finish. If the speed is slower than desired, make adjustments to make the Encryption Crawler faster as shown in this process. Please note the Encryption Crawler processes the data containers in reverse order from new to old. It's possible to back up new data after encryption is enforced but before the Encryption Crawler is turned on. If that happens, the Conversion Ratio could be less than 99% for the new data containers at the beginning. While the process is active, the value of Conversion Ratio can become higher with the fact that the older data containers can potentially have more unencrypted data. In this case, the Conversion Ratio, Containers Converted, and Containers Estimated can give more hints to determine the speed for these data containers. Monitoring the change of Conversion Ratio can give some hints on the proportion of the unencrypted data while the Encryption Crawler is active.
Tune the Graceful mode to run faster.	You can use the information in Tuning the Graceful mode to speed up the Graceful mode. After the initial tuning, you may need to check the progress and the system effect for the active jobs occasionally. You can do further tuning at any point during the process if desired. If the tuned Graceful mode negatively affects the system performance for the active jobs, you can consider turning off the Encryption Crawler for some of the MSDP partitions. You can keep it running for other partitions by following the recommendations in Turn on Encryption Crawler for part of the MSDP partitions to reduce system effect to reduce the system effect. You can also consider turning off the DataStore Write permission for some MSDP partitions by following the recommendations in Selectively disable DataStore Write for the MSDP partitions to reduce system effect which have the Encryption Crawler running. If the processing speed doesn't meet the expectations, the Aggressive mode can be leveraged for your environment.
Turn on the Aggressive mode.	You can use the information in Tuning the Aggressive mode to have the best performance for the Encryption Crawler. Veritas recommends that you start from the lowest level 2, then gradually increase to a higher level. You may need to check the progress and the system effect for the active jobs occasionally. You can perform further tuning at any point during the process if desired.
Find the tuning point which best balances the process speed and the system effect.	A faster Encryption Crawler speed usually means more effect on the system for all active jobs. A combination of tuning options may contribute a good balance between both.

More Information

Advanced options

Managing the Encryption Crawler

About the two modes of the Encryption Crawler

Managing the Encryption Crawler