Veritas NetBackup™ Deduplication Guide
- Introducing the NetBackup media server deduplication option
- Planning your deployment
- About MSDP storage and connectivity requirements
- About NetBackup media server deduplication
- About NetBackup Client Direct deduplication
- About MSDP remote office client deduplication
- About MSDP performance
- About MSDP stream handlers
- MSDP deployment best practices
- Provisioning the storage
- Licensing deduplication
- Configuring deduplication
- Configuring the Deduplication Multi-Threaded Agent behavior
- Configuring the MSDP fingerprint cache behavior
- Configuring MSDP fingerprint cache seeding on the storage server
- About MSDP Encryption using NetBackup KMS service
- Configuring a storage server for a Media Server Deduplication Pool
- Configuring a disk pool for deduplication
- Configuring a Media Server Deduplication Pool storage unit
- About MSDP optimized duplication within the same domain
- Configuring MSDP optimized duplication within the same NetBackup domain
- Configuring MSDP replication to a different NetBackup domain
- About NetBackup Auto Image Replication
- Configuring a target for MSDP replication to a remote domain
- Creating a storage lifecycle policy
- Resilient Network properties
- Editing the MSDP pd.conf file
- About protecting the MSDP catalog
- Configuring an MSDP catalog backup
- About NetBackup WORM storage support for immutable and indelible data
- Configuring deduplication to the cloud with NetBackup Cloud Catalyst
- Using NetBackup Cloud Catalyst to upload deduplicated data to the cloud
- Configuring a Cloud Catalyst storage server for deduplication to the cloud
- MSDP cloud support
- About MSDP cloud support
- Monitoring deduplication activity
- Viewing MSDP job details
- Managing deduplication
- Managing MSDP servers
- Managing NetBackup Deduplication Engine credentials
- Managing Media Server Deduplication Pools
- Changing a Media Server Deduplication Pool properties
- Configuring MSDP data integrity checking behavior
- About MSDP storage rebasing
- Managing MSDP servers
- Recovering MSDP
- Replacing MSDP hosts
- Uninstalling MSDP
- Deduplication architecture
- Troubleshooting
- About unified logging
- About legacy logging
- Troubleshooting MSDP installation issues
- Troubleshooting MSDP configuration issues
- Troubleshooting MSDP operational issues
- Troubleshooting Cloud Catalyst issues
- Cloud Catalyst logs
- Problems encountered while using the Cloud Storage Server Configuration Wizard
- Disk pool problems
- Problems during cloud storage server configuration
- Cloud Catalyst troubleshooting tools
- Trouble shooting multi-domain issues
- Appendix A. Migrating to MSDP storage
MSDP pd.conf file parameters
Table: pd.conf file parameters describes the deduplication parameters that you can configure for a NetBackup environment.
The parameters in this table are in alphabetical order; the parameters in a pd.conf
file may not be in alphabetical order.
The parameters in the file in your release may differ from those that are described in this topic.
You can edit the file to configure advanced settings for a host. If a parameter does not exist in a pd.conf
file, you can add it. During upgrades, NetBackup adds only required parameters to existing pd.conf
files.
The pd.conf file resides in the following directories:
(Windows) install_path\Veritas\NetBackup\bin\ost-plugins
(UNIX) /usr/openv/lib/ost-plugins/
Table: pd.conf file parameters
Parameter | Description |
---|---|
BACKUPRESTORERANGE |
On a client, specifies the IP address or range of addresses that the local network interface card (NIC) should use for backups and restores. Specify the value in one of two ways, as follows:
Default value: BACKUPRESTORERANGE= (no default value) Possible values: Classless Inter-Domain Routing format notation or comma-separated list of IP addresses |
BANDWIDTH_LIMIT |
Determines the maximum bandwidth that is allowed when backing up or restoring data between the deduplication host and the deduplication pool. The value is specified in KBytes/second. The default is no limit. Default value: BANDWIDTH_LIMIT = 0 Possible values: 0 (no limit) to the practical system limit, in KBs/sec |
COMPRESSION |
Specifies whether to compress the data during backups. By default, the data is compressed. Default value: COMPRESSION = 1 Possible values: 0 (off) or 1 (on) |
CR_STATS_TIMER |
Specifies a time interval in seconds for retrieving statistics from the storage server host. The default value of 0 disables caching and retrieves statistics on demand. Consider the following information before you change this setting:
Default value: CR_STATS_TIMER = 0 Possible values: 0 or greater, in seconds |
DEBUGLOG |
Specifies the file to which NetBackup writes the deduplication plug-in log information. NetBackup prepends a date stamp to each day's log file. On Windows, a partition identifier and slash must precede the file name. On UNIX, a slash must precede the file name. Note: This parameter does not apply for NDMP backups from a NetApp appliance. Default value:
Possible values: Any path |
DISABLE_BACKLEVEL_TLS |
When secure communication is established between the client and the server, this parameter specifies whether or not to disable older TLS versions. NetBackup version 8.0 and earlier use older TLS versions such as SSLV2, SSLV3, TLS 1.0, and TLS 1.1. To enable TLS 1.2, change the value of the DISABLE_BACKLEVEL_TLS parameter to 1 and restart the NetBackup Deduplication Engine (spoold) and the NetBackup Deduplication Manager (spad). Default value: DISABLE_BACKLEVEL_TLS = 0 Possible values: 0 (off) or 1 (on) Note: To enable TLS 1.2, NetBackup version must be 8.1 and later. When TLS 1.2 is enabled (DISABLE_BACKLEVEL_TLS = 1) on a machine (which can be a client or a media server or a load balance server), to establish communication, all machines connected to it must also enable TLS 1.2. For a standard backup, NetBackup client version 8.0 and earlier can communicate with NetBackup server (media server or load balance server) version 8.1 that has TLS 1.2 enabled. However, in case of optimized duplication and replication, load balance, and client direct duplication, NetBackup client versions 8.0 and earlier cannot communicate with NetBackup server (media server or load balance server) version 8.1, which has TLS 1.2 enabled. Further, NetBackup 5000 series appliance does not support TLS 1.2. |
DONT_SEGMENT_TYPES |
A comma-separated list of file name extensions of files not to be deduplicated. Files in the backup stream that have the specified extensions are given a single segment if smaller than 16 MB. Larger files are deduplicated using the maximum 16-MB segment size. Example: DONT_SEGMENT_TYPES = mp3,avi This setting prevents NetBackup from analyzing and managing segments within the file types that do not deduplicate globally. Note: this parameter does not apply to the NDMP backups that use the NetApp stream handler. Default value: DONT_SEGMENT_TYPES = (no default value) Possible values: comma-separated file extensions |
ENCRYPTION |
Specifies whether to encrypt the data during backups. By default, files are not encrypted. If you set this parameter to 1 on all hosts, the data is encrypted during transfer and on the storage. Default value: ENCRYPTION = 0 Possible values: 0 (no encryption) or 1 (encryption) |
FIBRECHANNEL |
Enable Fibre Channel for backup and restore traffic to and from a NetBackup series appliance. Default value: FIBRECHANNEL = 0 Possible values: 0 (off) or 1 (on) |
FILE_KEEP_ALIVE_INTERVAL |
The interval in seconds at which to perform keepalives on idle sockets. The following items describe the behavior based on how you configure this parameter:
Default value : FILE_KEEP_ALIVE_INTERVAL = 1440 Possible values: 0 (disabled) or 60 to 7200 seconds To determine the keep alive interval that NetBackup uses, examine the deduplication plug-in log file for a message similar to the following: Using keepalive interval of xxxx seconds For more information about the deduplication plug-in log file, see DEBUGLOG and LOGLEVEL in this table. |
FP_CACHE_CLIENT_POLICY |
Note: Veritas recommends that you use this setting on the individual clients that back up their own data (client-side deduplication). If you use it on a storage server or load balancing server, it affects all backup jobs. Specifies the client, backup policy, and date from which to obtain the fingerprint cache for the first backup of a client. By default, the fingerprints from the previous backup are loaded. This parameter lets you load the fingerprint cache from another, similar backup. It can reduce the amount of time that is required for the first backup of a client. This parameter especially useful for remote office backups to a central datacenter in which data travels long distances over a WAN. Specify the setting in the following format: clienthostmachine,backuppolicy,date The date is the last date in mm/dd/yyyy format to use the fingerprint cache from the client you specify. Default value: FP_CACHE_CLIENT_POLICY = (no default value) See Configuring MSDP fingerprint cache seeding on the client. |
FP_CACHE_INCREMENTAL |
Specifies whether to use fingerprint caching for incremental backups. Because incremental backups only back up what has changed since the last backup, cache loading has little affect on backup performance for incremental backups. Default value: FP_CACHE_INCREMENTAL = 0 Possible values: 0 (off) or 1 (on) Note: Change this value only when directed to do so by a Veritas representative. |
FP_CACHE_LOCAL |
Specifies whether or not to use the fingerprint cache for the backup jobs that are deduplicated on the storage server. This parameter does not apply to load balancing servers or to clients that deduplicate their own data. When the deduplication job is on the same host as the NetBackup Deduplication Engine, disabling the fingerprint cache improves performance. Default value: FP_CACHE_LOCAL = 1 Possible values: 0 (off) or 1 (on) |
FP_CACHE_MAX_COUNT |
Specifies the maximum number of images to load in the fingerprint cache. Default value: FP_CACHE_MAX_COUNT = 1024 Possible values: 0 to 4096 Note: Change this value only when directed to do so by a Veritas representative. |
FP_CACHE_MAX_MBSIZE |
Specifies the amount of memory in MBs to use for the fingerprint cache. Default value: FP_CACHE_MAX_MBSIZE = 20 Possible values: 0 to the computer limit Note: Change this value only when directed to do so by a Veritas representative. |
FP_CACHE_PERIOD_REBASING_THRESHOLD |
Specifies the threshold (MB) for periodic rebasing during backups. A container is considered for rebasing if both of the following are true:
Default value: FP_CACHE_PERIOD_REBASING_THRESHOLD = 16 Possible values: 0 (disabled) to 256 |
FP_CACHE_REBASING_THRESHOLD |
Specifies the threshold (MB) for normal rebasing during backups. A container is considered for rebasing if both of the following are true:
Default value:FP_CACHE_REBASING_THRESHOLD = 4 Possible values: 0 (disabled) to 200 If you change this value, consider the new value carefully. If you set it too large, all containers become eligible for rebasing. Deduplication rates are lower for the backup jobs that perform rebasing. |
LOCAL_SETTINGS |
Specifies whether to use the
To use the local settings, set this value to 1. Default value: LOCAL_SETTINGS = 0 Possible values: 0 (allow override) or 1 (always use local settings) |
LOGLEVEL |
Specifies the amount of information that is written to the log file. The range is from 0 to 10, with 10 being the most logging. Default value: LOGLEVEL = 0 Possible values: An integer, 0 to 10 inclusive Note: Change this value only when directed to do so by a Veritas representative. |
MAX_IMG_MBSIZE |
The maximum backup image fragment size in megabytes. Default value: MAX_IMG_MBSIZE = 51200 Possible values: 0 to 51,200, in MBs Note: Change this value only when directed to do so by a Veritas representative. |
MAX_LOG_MBSIZE |
The maximum size of the log file in megabytes. NetBackup creates a new log file when the log file reaches this limit. NetBackup prepends the date and the ordinal number beginning with 0 to each log file, such as Default value: MAX_LOG_MBSIZE = 100 Possible values: 0 to 50,000, in MBs |
META_SEGKSIZE |
The segment size for metadata streams Default value: META_SEGKSIZE = 16384 Possible values: 32-16384, multiples of 32 Note: Change this value only when directed to do so by a Veritas representative. |
MTSTRM_BACKUP_CLIENTS |
If set, limits the use of the Multi-Threaded Agent to the backups of the specified clients. The clients that are not specified use single-threading. This setting does not guarantee that the specified clients use the Multi-Threaded Agent. The MaxConcurrentSessions parameter in the See MSDP mtstrm.conf file parameters. The format is a comma-separated list of the clients, case insensitive (for example, MTSTRM_BACKUP_CLIENTS = client1,client2,client3). Default value: MTSTRM_BACKUP_CLIENTS = (no default value) Possible values: comma separated client names |
MTSTRM_BACKUP_ENABLED |
Use the Multi-Threaded Agent in the backup stream between the deduplication plug-in and the NetBackup Deduplication Engine. Default value: MTSTRM_BACKUP_ENABLED = (no default value) Possible values: 1 (On) or 0 (Off) The value for this parameter is configured during installation or upgrade. If the hardware concurrency value of the host is greater than a hardware concurrency threshold value, NetBackup sets MTSTRM_BACKUP_ENABLED to 1. (For the purposes of this parameter, the hardware concurrency is the number of CPUs or cores or hyperthreading units.) The following items describe the values that are used for the determination algorithm:
The following examples may be helpful:
|
MTSTRM_BACKUP_POLICIES |
If set, limits the use of the Multi-Threaded Agent to the backups of the specified policies. The clients in the policies that are not specified use single-threading, unless the client is specified in the MTSTRM_BACKUP_CLIENTS parameter. This setting does not guarantee that all of the clients in the specified policies use the Multi-Threaded Agent. The MaxConcurrentSessions parameter in the See MSDP mtstrm.conf file parameters. The format is a comma-separated list of the policies, case sensitive (for example, MTSTRM_BACKUP_POLICIES = policy1,policy2,policy3). Default value: MTSTRM_BACKUP_POLICIES = (no default value) Possible values: comma separated backup policy names |
MTSTRM_IPC_TIMEOUT |
The number of seconds to wait for responses from the Multi-Threaded Agent before the deduplication plug-in times out with an error. Default value: MTSTRM_IPC_TIMEOUT = 1200 Possible values: 1-86400, inclusive |
OPTDUP_BANDWIDTH |
Determines the bandwidth that is allowed for each optimized duplication and Auto Image Replication stream on a deduplication server. OPTDUP_BANDWIDTH does not apply to clients. The value is specified in KBytes/second. Default value: OPTDUP_BANDWIDTH= 0 Possible values: 0 (no limit) to the practical system limit, in KBs/sec A global bandwidth parameter effects whether or not OPTDUP_BANDWIDTH applies. See About configuring MSDP optimized duplication and replication bandwidth. |
OPTDUP_COMPRESSION |
Specifies whether to compress the data during optimized duplication and Auto Image Replication. By default, files are compressed. To disable compression, change the value to 0. This parameter does not apply to clients. Default value: OPTDUP_COMPRESSION = 1 Possible values: 0 (off) or 1 (on) |
OPTDUP_ENCRYPTION |
Specifies whether to encrypt the data during optimized duplication and replication. By default, files are not encrypted. If you want encryption, change the value to 1 on the MSDP storage server and on the MSDP load balancing servers. This parameter does not apply to clients. If you set this parameter to 1 on all hosts, the data is encrypted during transfer. Default value: OPTDUP_ENCRYPTION = 0 Possible values: 0 (off) or 1 (on) |
OPTDUP_TIMEOUT |
Specifies the number of minutes before the optimized duplication times out. Default value: OPTDUP_TIMEOUT = 720 Possible values: The value, expressed in minutes |
PDDO_SSL_VERIFY |
Specifies whether to perform SSL verification when communicating with the NetBackup 5000 series appliance. By default, SSL verification is not performed while communication with the NetBackup 5000 series appliance. To perform SSL verification, change the value to 1 and copy the root certificate file of NetBackup 5000 series appliance to following directory:
The root certificate file on NetBackup 5000 series appliance is:
If there are multiple NetBackup servers and NetBackup clients that communicate with NetBackup 5000 series appliance, the root certificate file must be copied to all the media servers and NetBackup clients. Default value: PDDO_SSL_VERIFY = 0 Possible values: 0 (off) or 1 (on) |
PREFERRED_EXT_SEGKSIZE |
Specifies the file extensions and the preferred segment sizes in KB for specific file types. File extensions are case sensitive. The following describe the default values: edb are Exchange Server files; mdfare SQL Server master database files, ndf are SQL Server secondary data files, and segsize64k are Microsoft SQL streams. Default value: PREFERRED_EXT_SEGKSIZE = edb:32,mdf:64,ndf:64,segsize64k:64 Possible values: file_extension:segment_size_in_KBs pairs, separated by commas. See also SEGKSIZE. |
PREFETCH_SIZE |
The size in bytes to use for the data buffer for restore operations. Default value: PREFETCH_SIZE = 33554432 Possible values: 0 to the computer's memory limit Note: Change this value only when directed to do so by a Veritas representative. |
|
Specifies on which host to decrypt and decompress the data during restore operations. Depending on your environment, decryption and decompression on the client may provide better performance.
Default value: Possible values: 0 enables decryption and decompression on the media server; 1 enables decryption and decompression on the client. |
SEGKSIZE |
The default file segment size in kilobytes. Default value: SEGKSIZE = 128 Possible values: 32 to 16384 KBs, increments of 32 only Warning: Changing this value may reduce capacity and decrease performance. Change this value only when directed to do so by a Veritas representative. You can also specify the segment size for specific file types. See PREFERRED_EXT_SEGKSIZE. |
VLD_CLIENT_NAME |
Specifies the name of the NetBackup client to enable variable-length deduplication. By default, the VLD_CLIENT_NAME parameter is not present in the You can also specify different maximum and minimum segment sizes with this parameter for different NetBackup clients. If you do not specify the segment sizes, then the default values are considered. The values are case-sensitive. Use in any of the following formats:
Note: You can add a maximum of 50 clients in the |
VLD_MIN_SEGKSIZE |
The minimum size of the data segment for variable-length deduplication in KB. The segment size must be in multiples of 4 and fall in between 4 KB to 16384 KB. The default value is 64 KB. The value must be smaller than VLD_MAX_SEGKSIZE. Different NetBackup clients can have different segment sizes. A larger value reduces the CPU consumption, but decreases the deduplication ratio. A smaller value increases the CPU consumption, but increases the deduplication ratio Note: Keeping similar or close values for VLD_MIN_SEGKSIZE and VLD_MAX_SEGKSIZE results in a performance that is similar to fixed-length deduplication. |
VLD_MAX_SEGKSIZE |
The maximum size of the data segment for variable-length deduplication in KB. VLD_MAX_SEGKSIZE is used to set a boundary for the data segments. The segment size must be in multiples of 4 and fall in between 4 KB to 16384 KB. The default value is 128 KB. The value must be greater than VLD_MIN_SEGKSIZE. Different NetBackup clients can have different segment sizes. Note: Keeping similar or close values for VLD_MIN_SEGKSIZE and VLD_MAX_SEGKSIZE results in a performance that is similar to fixed-length deduplication. |
VLD_POLICY_NAME |
Specifies the name of the backup policy to enable variable-length deduplication. By default, the VLD_POLICY_NAME parameter is not present in the pd.conf configuration file. You can also specify different maximum and minimum segment sizes with this parameter for different NetBackup policies. If you do not specify the segment sizes, then the default values are considered. The values are case-sensitive. Use in any of the following formats:
Note: You can add a maximum of 50 policies in the |