NetBackup™ Deduplication Guide

Command usage example outputs

When encryption is not enforced or the rolling data conversion is not finished, the crcontrol command denies Encryption Crawler related operations. The following is an example of the output:

[root@rsvlmvc01vm0771 /]# /usr/openv/pdde/pdcr/bin/crcontrol --encconvertstate
CRControlEncConvertInfoGet failed : operation not supported
Please double check the server encryption settings

Check the data format of a data container before the Encryption Crawler process. The following is an example of the output:

[root@rsvlmvc01vm0771 /]# /usr/openv/pdde/pdcr/bin/dcscan --so-data-format 3080|head -n 15
Path = /MSDP/data/3/3080.[bhd, bin]
*** Header for container 3080 ***
version                    : 1
flags                      : 0xe000(DC_ENTRY_FULL|DC_ENTRY_SHA256|DC_ENTRY_BINHEADER)
data file last position    : 67001810
header file last position  : 55252
source id                  : 2505958
retention                  : 0
file size                  : 67001810
delete space               : 0
active records             : 511
total records              : 511
deleted records            : 0
crc32                      : 0x4fd80a49

[root@rsvlmvc01vm0771 /]# /usr/openv/pdde/pdcr/bin/dcscan --so-data-format 3080|tail -n 15
type of record : SO
version        : 4
flags          : 0x2
backup session : 1670238781
fptype         : 3
size           : 131118
record crc     : 4164163489
data crc       : 1313121942
ctime          : 1642086781
offset         : 66870692
digest         : 7f7fd0c5d8fc64d9a7e25c7c079af86613b40d9feff9d316cdfc09c1eafb1690
KMS Enc        : NO
SO crc         : 85135236
data format    : [LZO Compressed Streamable, v2, window size 143360 bytes]

[root@rsvlmvc01vm0771 /]# /usr/openv/pdde/pdcr/bin/dcscan 
--so-data-format 3080|grep "data format"|wc
    511    5621   38325
[root@rsvlmvc01vm0771 /]# /usr/openv/pdde/pdcr/bin/dcscan 
--so-data-format 3080|grep "data format"|tail -n 5
data format    : [LZO Compressed Streamable, v2, window size 143360 bytes]
data format    : [LZO Compressed Streamable, v2, window size 143360 bytes]
data format    : [LZO Compressed Streamable, v2, window size 143360 bytes]
data format    : [LZO Compressed Streamable, v2, window size 143360 bytes]
data format    : [LZO Compressed Streamable, v2, window size 143360 bytes]
[root@rsvlmvc01vm0771 /]# /usr/openv/pdde/pdcr/bin/dcscan 
--so-data-format 3080|grep "data format"|grep -i -e "AES" -e "Encrypted"

Check the data format of a data container after the Encryption Crawler process. The following is an example of the output:

[root@rsvlmvc01vm0771 /]# /usr/openv/pdde/pdcr/bin/dcscan --so-data-format 3080|head -n 15
Path = /MSDP/data/3/3080.[bhd, bin]
*** Header for container 3080 ***
version                    : 1
flags                      : 0xe000(DC_ENTRY_FULL|DC_ENTRY_SHA256|DC_ENTRY_BINHEADER)
data file last position    : 67009986
header file last position  : 55252
source id                  : 2505958
retention                  : 0
file size                  : 67009986
delete space               : 0
active records             : 511
total records              : 511
deleted records            : 0
crc32                      : 0x54380a69

[root@rsvlmvc01vm0771 /]# /usr/openv/pdde/pdcr/bin/dcscan --so-data-format 3080|tail -n 15
type of record : SO
version        : 4
flags          : 0x2
backup session : 1670238781
fptype         : 3
size           : 131134
record crc     : 4210300849
data crc       : 1992124019
ctime          : 1642086781
offset         : 66878852
digest         : 7f7fd0c5d8fc64d9a7e25c7c079af86613b40d9feff9d316cdfc09c1eafb1690
KMS Enc        : NO
SO crc         : 85331847
data format    : [AES-256-CTR Encrypted archive 256bit key LZO Compressed Streamable, 
v2, window size 143360 bytes]

[root@rsvlmvc01vm0771 /]# /usr/openv/pdde/pdcr/bin/dcscan 
--so-data-format 3080|grep "data format"|wc
    511    8176   59276

[root@rsvlmvc01vm0771 /]# /usr/openv/pdde/pdcr/bin/dcscan 
--so-data-format 3080|grep "data format"|tail -n 5
data format    : [AES-256-CTR Encrypted archive 256bit key LZO Compressed Streamable, 
v2, window size 143360 bytes]
data format    : [AES-256-CTR Encrypted archive 256bit key LZO Compressed Streamable, 
v2, window size 143360 bytes]
data format    : [AES-256-CTR Encrypted archive 256bit key LZO Compressed Streamable, 
v2, window size 143360 bytes]
data format    : [AES-256-CTR Encrypted archive 256bit key LZO Compressed Streamable, 
v2, window size 143360 bytes]
data format    : [AES-256-CTR Encrypted archive 256bit key LZO Compressed Streamable, 
v2, window size 143360 bytes]

[root@rsvlmvc01vm0771 /]# /usr/openv/pdde/pdcr/bin/dcscan 
--so-data-format 3080|grep "data format"|grep -i -e "AES" -e "Encrypted"
data format    : [AES-256-CTR Encrypted archive 256bit key LZO Compressed Streamable, 
v2, window size 143360 bytes]
data format    : [AES-256-CTR Encrypted archive 256bit key LZO Compressed Streamable, 
v2, window size 143360 bytes]
data format    : [AES-256-CTR Encrypted archive 256bit key LZO Compressed Streamable, 
v2, window size 143360 bytes]

[root@rsvlmvc01vm0771 /]# /usr/openv/pdde/pdcr/bin/dcscan --so-is-encrypted 3080
1 of 1: unencrypted 0: container 3080: size 67009986

Using dcscan --so-is-encrypted to check if a container or a list of containers are encrypted.

The status message unencrypted 0 indicate it's encrypted already, and unencrypted 1 indicates it's unencrypted and needs to be encrypted. The following is an example of the output:

[root@rsvlmvc01vm0771 /]# /usr/openv/pdde/pdcr/bin/dcscan --so-is-encrypted 3080
1 of 1: unencrypted 1: container 3080: size 67001810

Veritas recommends using the reporting tool encryption_reporting to report the unencrypted data in the MSDP pool.

By default, the reporting tool creates a thread pool of two threads. The tool uses these threads to search for unencrypted data or to encrypt the unencrypted data. A thread is used to process one MSDP mount point to completion. Upon completing the processing of a mount point, the thread is returned to the thread pool. The thread is then used to process any additional mount point that is queued up for processing.

The number of threads is equal to the number of mountpoints that can be processed concurrently. You can increase or decrease the thread pool's thread count by specifying the -n option. The minimum thread count is 1 and the maximum is 20.

The reporting tool is I/O intensive. Increasing the thread count up to the total number of MSDP mountpoints usually means better performance for the reporting tool. It also means more load on the system which can affect performance of backup, restore, deduplication, and replication jobs. No performance gains are observed for using more threads than there are mountpoints.

When using the reporting tool to search for the unencrypted data, each thread invokes one instance of dcscan. Each dcscan instance uses roughly N * 160 MB of memory. In this equation, N is the number of MSDP mountpoints on the server. If there are a total of 12 MSDP mountpoints, each dcscan instance uses about 1.8 GB of memory. If there are four threads running in the reporting tool, the reporting tool and the dcscan processes consume more than 7 GB of memory.

On a Windows BYO, the default path to dcscan is C:\Program Files\Veritas\pdde. If you have dcscan installed somewhere else, you must use the -d or --dcscan_dir option to specify the correct location.

The encryption_reporting does not account for data encrypted with the Encryption Crawler. If you have previously run the Encryption Crawler to encrypt data, you must clear the metadata files with the -c option if they exist. Then re-run encryption_reporting to get up-to-date information.

In certain circumstances, data may be reported as Encrypted needs KMS convert. This means that the data is encrypted, but not with KMS. If you see this message, use the crawler commands ./crcontrol - encconvertreset and ./crcontrol - encconverton to encrypt the rest of the data with KMS.

Veritas does not recommend that you run the reporting tool while the Encryption Crawler process is active.