NetBackup™ Deduplication Guide

The following list describes the backup process for an MSDP client to a Media Server Deduplication Pool:

• The NetBackup Job Manager (nbjm) starts the Backup/Restore Manager (bpbrm) on a media server.

• The Backup/Restore Manager probes the client to determine if it is configured and ready for deduplication.

• If the client is ready, the Backup/Restore Manager starts the following processes: The OpenStorage proxy server (nbostpxy) on the client and the data moving processes (bpbkar) on the client and bptm on the media server).

  NetBackup uses the proxy plug-in on the media server to route control information from bptm to nbostpxy.

• The Backup/Archive Manager (bpbkar) generates the backup images and moves them to the client nbostpxy process by shared memory.

  The Backup/Archive Manager also sends the information about files within the image to the Backup/Restore Manager (bpbrm). The Backup/Restore Manager sends the file information to the bpdbm process on the master server for the NetBackup database.

• The client nbostpxy process moves the data to the deduplication plug-in.

• The deduplication plug-in on the client tries to retrieve a list of fingerprints, in the following order:

  • From a client and a policy that is configured in the client's pd.conf file. The FP_CACHE_CLIENT_POLICY entry defines the client and policy to use for the fingerprint cache. The entry must be valid (that is, not expired).

    See About seeding the MSDP fingerprint cache for remote client deduplication.

  • From the previous backup for the client and policy.

  • From the special seeding directory on the storage server.

    See About seeding the MSDP fingerprint cache for remote client deduplication.

  The list of fingerprints is used as a cache so the plug-in does not have to request each fingerprint from the engine.

  If no fingerprints are loaded into the cache, the deduplication rate may be very low for the backup.

• The deduplication plug-in separates the files in the backup image into segments.

• The deduplication plug-in buffers the segments and then sends batches of them to the Deduplication Multi-Threaded Agent. Multiple threads and shared memory are used for the data transfer.

• The NetBackup Deduplication Multi-Threaded Agent processes the data segments in parallel using multiple threads to improve throughput performance. The agent then sends only the unique data segments to the NetBackupDeduplication Engine.

• The NetBackup Deduplication Engine writes the data to the Media Server Deduplication Pool.

  The first backup may have a 0% deduplication rate, although a 0% deduplication rate is unlikely. Zero percent means that all file segments in the backup data are unique.