Search <book_title>...

NetBackup™ Deduplication Guide

Last Published: 2022-03-28

Product(s): NetBackup (10.0)

Managing the Encryption Crawler

Use the crcontrol command to manage the Encryption Crawler. The following table describes the options you can use to manage how the Encryption Crawler functions.

Table: crcontrol command options

Option	Description
--encconverton	To enable and start the Encryption Crawler process, use *--encconverton [num]. The num* variable is optional and indicates the number for the partition index (starting from 1). The parameter enables the Encryption Crawler for the specified MSDP partition. If *num* is not specified, it is enabled for all MSDP partitions. The *num* variable is not supported on a BYO setup when the `/etc/nbapp-release` (Linux), or `c:\etc\nbapp-release` (Windows) file isn't present. On a BYO setup, create the file to enable multiple volumes support and then the *num* variable is supported.
--encconvertoff	To disable and stop the Encryption Crawler process, use *--encconvertoff [num]. The num* variable is optional and indicates the number for the partition index (starting from 1). The parameter enables the Encryption Crawler for the specified MSDP partition. If *num* is not specified, it disabled for all MSDP partitions. The *num* variable is not supported on a BYO setup when the `/etc/nbapp-release` (Linux), or `c:\etc\nbapp-release` (Windows) file isn't present. On a BYO setup, create the file to enable multiple volumes support and then the *num* variable is supported.
--encconvertlevel	To switch between Graceful mode and Aggressive mode, use --encconvertlevel level. The *level* is required. A value of 1 for the *level* variable is the default for Graceful mode. A value for the *level* variable that is between 2-4 indicates that Aggressive mode is enabled. A larger number indicates that the Encryption crawler is more aggressive.
--encconvertstate	To determine the mode of the Encryption Crawler process and the progress, use --encconvertstate [verbose]. Optionally, you can specify a verbose level (0-2) for this option. 0 is the default verbose level for the overall brief information. 1 is for the overall information and the details of each partition. 2 is for the overall information and the details of each partition. The details of a partition are shown even if the process is finished for the partition. The verbose parameter is not supported on a BYO setup when the `/etc/nbapp-release` (Linux), or `c:\etc\nbapp-release` (Windows) file isn't present. On a BYO setup, create the file to enable multiple volumes support and then the *num* variable is supported.

For more information about the crcontrol, refer to the following:

NetBackup Commands Reference Guide

Once the Encryption Crawler is turned on, you can monitor the status, mode, and progress with the crcontrol --encconvertstate command.

Table: Encryption Crawler monitor

Item	Description
Status	Shows if the Encryption Crawler is ON, OFF, or Finished.
Level	Shows in which level and mode the Encryption Crawler is. The value is in the format mode (level), for example Graceful (1).
Busy	Shows if the Encryption Crawler is busy or not.
Max Group ID	The maximum container group ID to process when the Encryption Crawler is turned on. It's the data boundary and doesn't change once Encryption Crawler is turned on.
Current Group ID	Currently processing this group ID.
Current Container ID	Currently processing this container ID.
Containers Estimated	The estimated number of data containers in the MSDP pool that the Encryption Crawler must process. It's a statistic information and there may be inaccuracy for performance reasons. Once the Encryption Crawler is turned on, the value is not updated.
Containers Scanned	The number of data containers the Encryption Crawler must process.
Containers Converted	The number of containers encrypted by the Encryption Crawler process.
Containers Skipped	The number of data containers that the Encryption Crawler skipped. The reasons vary and are described in About the skipped data containers. If there are skipped data containers, you can check the Encryption Crawler log or the history log for the details. The encryption_reporting tool may help report and encrypt the individual containers after the Encryption Crawler process finishes. Details about this encryption_reporting tool are available. See Encrypting the data. See Command usage example outputs.
Data Size Scanned	The aggregated data size of the scanned data containers for Containers Scanned.
Data Size Converted	The aggregated data size of the converted data containers for Containers Converted.
Progress	The proportion of the total estimated data containers that the Encryption Crawler has scanned. Progress = Containers Scanned / Containers Estimated
Conversion Ratio	The proportion of the scanned data size which the Encryption Crawler has converted. Conversion Ratio = Data Size Converted / Data Size Scanned
Mount Points Information	The status of each mount point. If a verbose value of 1 is specified for the --encconvertstate option, the details of the unfinished mount points are printed. If a verbose value of 2 is specified for --encconvertstate option, the details of all the mount points are printed regardless of completion state.

The Progress line in the log can be used to extrapolate how long the Encryption Crawler is expected to take. For example, if 3.3% of the pool is completed in 24 hours, the process may take about 30 days to finish.

Note:

The Encryption Crawler processes the data containers in reverse order from new to old.

It's possible to back up new data after encryption is enforced but before the Encryption Crawler is turned on. If that happens, the Conversion Ratio could be less than 99% for the new data containers at the beginning. While the process is running, the value of Conversion Ratio can become higher with the fact that the older data containers can potentially have more unencrypted data. In this case, the Conversion Ratio, Containers Converted, and Containers Estimated can help estimate the speed for these data containers.

Monitoring the change of Conversion Ratio can give some indication for the proportion of the unencrypted data while the Encryption Crawler is active.

Note:

During the encryption process, the progress survives in the case of MSDP restart.

About the skipped data containers

The reasons the Encryption Crawler skips some data containers as reported by Containers Skipped include:

If a data container is to be expired but not yet deleted, it is skipped.
If a data container has a possible data integrity issue, it is skipped. The Encryption Crawler conveys the container to the CRC check process to identify and possibly fix the container.
If Instant Access or Universal Share is configured, and if some shares are not checkpointed before the Encryption Crawler process, the shares may hold some data containers with exclusive permission. Those data containers are skipped. Veritas recommends that you create checkpoints for all the shares of Instant Access or Universal Share before turning on the Encryption Crawler process. By doing so, VpFS releases the exclusive permission of those data containers for spoold and the Encryption Crawler to process them.
Appliances starting with the release of 3.1.2 can have empty data containers the VpFS root share vpfs0 reserves, even if Instant Access or Universal Share is configured. This situation can also occur on a BYO setup where Instant Access or Universal Share is configured. Normally, VpFS does not release the exclusive permission of those data containers. Those data containers are skipped. You can ignore these skipped containers.
Here how to check if the skipped data containers are empty and if the VpFS root share vpfs0 owns them. You can check the other VpFS owned data containers in the similar way.
- You can find the skipped data containers that are identified as owned by VpFS in the Encryption Crawler log by looking for the following:
  n152-h21:/home/maintenance # grep VpFS /msdp/data/dp1/pdvol/log/spoold/enccrawler.log
  February 04 05:13:14 WARNING [139931343951616]: -1: __getDcidListFromOneGroup: 1 containers owned by VpFS in group 7 were skipped. min DC ID 7168, max DC ID 7168
- Check if the VpFS root share vpfs0 owns the data containers.
```
n152-h21:/home/maintenance # cat /msdp/data/dp1/4pdvol/7/.shareid
vpfs0
106627568
```

The data containers that the VpFS root share vpfs0 owns, are empty.

n152-h21:/home/maintenance # ls -Al /msdp/data/dp1/4pdvol/7
total 24
-rw-r--r-- 1 root root 64 Feb 1 02:40 7168.bhd
-rw-r--r-- 1 root root 0 Feb 1 02:40 7168.bin
-rw------- 1 root root 12 Feb 1 02:40 .dcidboundary
-rw-r----- 1 root root 15 Feb 1 02:40 .shareid
drwxr-xr-x 3 root root 96 Feb 4 15:37 var
n152-h21:/home/maintenance # /usr/openv/pdde/pdcr/bin/dcscan 7168
Path = /msdp/data/dp1/4pdvol/7/7168.[bhd, bin]
*** Header for container 7168 ***
version : 1
flags : 0x4000(DC_ENTRY_SHA256)
data file last position : 0
header file last position : 64
source id : 0
retention : 0
file size : 0
delete space : 0
active records : 0
total records : 0
deleted records : 0
crc32 : 0x1d74009d

More Information

About provisioning the storage for MSDP