NetBackup™ Deployment Guide for Azure Kubernetes Services (AKS) Cluster
- Introduction to NetBackup on AKS
- Deployment with environment operators
- Assessing cluster configuration before deployment
- Deploying NetBackup
- Preparing the environment for NetBackup installation on AKS
- Recommendations of NetBackup deployment on AKS
- Limitations of NetBackup deployment on AKS
- About primary server CR and media server CR
- Monitoring the status of the CRs
- Updating the CRs
- Deleting the CRs
- Configuring NetBackup IT Analytics for NetBackup deployment
- Managing NetBackup deployment using VxUpdate
- Migrating the node pool for primary or media servers
- Deploying MSDP Scaleout
- Monitoring NetBackup
- Monitoring MSDP Scaleout
- Managing the Load Balancer service
- Performing catalog backup and recovery
- Managing MSDP Scaleout
- About MSDP Scaleout maintenance
- Uninstalling MSDP Scaleout from AKS
- Troubleshooting
- View the list of operator resources
- View the list of product resources
- View operator logs
- View primary logs
- Pod restart failure due to liveness probe time-out
- Socket connection failure
- Resolving an invalid license key issue
- Resolving an issue where external IP address is not assigned to a NetBackup server's load balancer services
- Resolving the issue where the NetBackup server pod is not scheduled for long time
- Resolving an issue where the Storage class does not exist
- Resolving an issue where the primary server or media server deployment does not proceed
- Resolving an issue of failed probes
- Resolving token issues
- Resolving an issue related to insufficient storage
- Resolving an issue related to invalid nodepool
- Resolving a token expiry issue
- Resolve an issue related to inconsistency in file ownership
- Resolve an issue related to KMS database
- Resolve an issue related to pulling an image from the container registry
- Resolving an issue related to recovery of data
- Check primary server status
- Pod status field shows as pending
- Ensure that the container is running the patched image
- Getting EEB information from an image, a running container, or persistent data
- Appendix A. CR template
Ensure that the container is running the patched image
There are three copies of the container image present in the Kubernetes environment during deployment or patching.
The first image copy is created on a local docker instance during image load operation. To check this copy, do the following:
- Run:
$ docker load -i images/pdk8soptr-16.0.tar.gz
Sample output:
Loaded image: msdp-operator:16.0 - Taking the image name from step 1, run:
$ docker image ls | grep msdp-operator
Sample output:
msdp-operator 16.0 353d2bd50105 2 days ago 480 MB - Taking the value from step 2, run:
$ docker inspect 353d2bd50105 | jq .[].Id
"sha256:353d2bd50105cbc3c61540e10cf32a152432d5173bb6318b8e"
The second copy is created in Azure Container Registry (ACR). To check this copy, do the following:
- Run:
$ docker image tag msdp-operator:16.0 testregistry.azurecr.io/msdp-operator:16.0
- Run:
$ docker image ls | grep msdp-operator
Sample output:
msdp-operator 16.0 353d2bd50105 2 days ago 480 MB tregistry.azurecr.io/msdp-operator 16.0 353d2bd50105 2 days ago 480 MB
- To push the image to the registry, run:
$ docker push testregistry.azurecr.io/msdp-operator
The push refers to a repository [testregistry.azurecr.io/msdp-operator]
0a504041c925: Layer already exists
16.0: digest: sha256:d294f260813599562eb5ace9e0acd91d61b7dbc53c3 size: 2622
- To verify local image digest after the push operation, run:
$ docker inspect 353d2bd50105 | jq .[].RepoDigests
Sample output:
[ "testregistry.azurecr.io/msdp-operator@sha256: d294f260813599562eb5ace9e0acd91d61b7dbc53c3" ]
- To verify image presence in the registry, run:
$ az acr repository list --name testregistry
Sample output:
[ "msdp-operator", ]
- To verify image digest in registry, run:
$ az acr repository show -n testregistry --image msdp-operator:16.0
Sample output:
{ "changeableAttributes": { "deleteEnabled": true, "listEnabled": true, "readEnabled": true, "writeEnabled": true }, "createdTime": "2022-02-01T13:43:26.6809388Z", "digest": "sha256:d294f260813599562eb5ace9e0acd91d61b7dbc53c3", "lastUpdateTime": "2022-02-01T13:43:26.6809388Z", "name": "16.0", "signed": false }
The third copy is located on a Kubernetes node running the container after it is pulled from the registry. To check this copy, do the following:
- Run;
$ kubectl get nodes -o wide
NAME STATUS VERSION INTERNAL-IP OS-IMAGE aks-agentpool-7601-vmss000 Ready v1.21.7 10.240.0.4 Ubuntu 18.04.6 LTS
- Use kubectl debug to run a container on the node:
$ kubectl debug node/aks-nodepool1-7601-vmss000-it --image=mcr.microsoft.com/aks/fundamental/base-ubuntu:v0.0.11 root@aks-agentpool-7601-vmss000:/#
- You can interact with the node session from the privileged container:
chroot /host
- Verify the presence of the image:
/usr/local/bin/crictl image | grep msdp
Sample output:
testregistry.azurecr.io/msdp-operator 16.0 353d2bd50105c 182MB
- Verify the image ID on the Kubernetes node, run:
/usr/local/bin/crictl inspecti 353d2bd50105c | jq .[].id
Sample output
"sha256:353d2bd50105cbc3c61540e10cf32a152432d5173bb6318b8e" null
- Verify the image digest on the Kubernetes node, run:
/usr/local/bin/crictl inspecti 353d2bd50105c | jq .[].repoDigests
Sample output
[ "testregistry.azurecr.io/msdp-operator@sha256: d294f260813599562eb5ace9e0acd91d61b7dbc53c3" ] null
Use the steps given above to identify image ID and Digest and compare with values obtained from the registry and the Kubernetes node running the container.
Note:
MSDP Scaleout images (uss-engine, uss-mds, uss-controller, msdp-operator) use IfNotPresent imagePullPolicy. A unique image tag is required in order for a Kubernetes node to pull an updated image.