Slow deduplication when using VDD on Access Appliance in a Docker/Podman Container

Problem

Slow deduplication when using VDD on Access Appliance (all models) with the MSDP/LTR processes running in a Docker/Podman container.
 

Error Message

Check the spad.log and spoold.log for entries similar to the following

October 28 10:37:59 WARNING [140681366587200]: -1: NetWaitForRemote: There might be DNS issue. It took 12.024 seconds to resolve remote address (::ffff:XXX.XXX.XXX.XXX, 39790).

Where XXX.XXX.XXX.XXX will be the IP address that is slow to resolve.

At version 8.0, the spad.log and spoold.log can be found at:

/opt/VRTSnas/log/dedupe_store_link/spad/spad.log
/opt/VRTSnas/log/dedupe_store_link/spoold/spoold.log

At version 8.1 and later there can be more than 1 VDD pool. Hence. the link name is changed to also include the VDD pool name:

/opt/VRTSnas/log/dedupe_store_link_msdp-engine/spad/spad.log
/opt/VRTSnas/log/dedupe_store_link_msdp-engine/spoold/spoold.log

Cause

The deduplication processes run within a docker container, which uses a different network routing compared to outside. This can slow down DNS resolution, depending on networking configuration.

Solution

In the Access Solutions Guide for Netbackup, it is recommended to add the FQDN (Fully Qualified Domain Name) of the affected hosts:

----

You can also add FQDNs to the deduplication server. This configuration leads to faster backups on the storage server and are required before the user adds the storage server to the NetBackup server.

To add an FQDN to the deduplication server

• To add an FQDN to the deduplication cluster, enter the following: dedupe> fqdn add <IP> <fqdn>

----

Find all IP addresses which are slow to resolve and manually add them to the VDD deduplication configuration as follows.

Note: do not add the FQDN and IP of the deduplication service itself - this is unnecessary and can lead to upgrade errors if/when upgrading to Access 8.2 or later.  Please see Veritas KB article 100065990 for further details.

In this example, there are 3 IP addresses which the spad.log says are slow to resolve.
Check all spad logs and all spoold logs for these slow DNS warnings.

$ cd /opt/VRTSnas/log/dedupe_store_link/spad/
$ grep "There might be DNS issue" spad.log*  | awk '{ print $21 }' | sort | uniq
(::ffff:192.168.1.1,
(::ffff:192.168.1.2,
(media2.veritas.com,

Repeat for spoold logs

$ cd /opt/VRTSnas/log/dedupe_store_link/spoold/
$ grep "There might be DNS issue" spoold.log*  | awk '{ print $21 }' | sort | uniq

Add the IP addresses to the VDD configuration using the Access clish

CLISH> dedupe fqdn add 192.168.1.1 master1.veritas.com
CLISH> dedupe fqdn add 192.168.1.2 media1.veritas.com
CLISH> dedupe fqdn add 192.168.1.3 media2.veritas.com

Note: If you need to add hostnames that do not have a FQDN (shortname only), please contact Veritas Support for assistance and reference internal-only KB article 100061453