Procedure to upgrade AKS or EKS cluster hosting CloudScale

Procedure to upgrade AKS or EKS cluster hosting Cloud Scale:

1. Before upgrading the kubernetes cluster, ensure that the NetBackup Catalog is successfully backed up on cloud LSU (MSDP-C).
2. Pause NetBackup job scheduling and allow all running jobs to be completed.
3. Stop primary services
   1. For NetBackup version 10.5 and above, run the script below with a stop option.
      /VRTSk8s-netbackup-<version>/scripts$ ./cloudscale_restart.sh stop
   2. For NetBackup version lower than 10.5:

      1. Get the environment name with the following command:
         kubectl get environment -n <namespace>

      2. Pause with the following command:
         kubectl patch environment <env_name> -n <namespace> --type=merge -p='{"spec": {"primary": {"paused": true}}}'

      3. Scale down the primary pod to 0 as follows:
         1. Get primary sts name:
            kubectl get sts -n <namespace> | grep "\-primary"
         2. Scale to 0:
            kubectl scale --replicas=0 sts <sts name> -n <namespace>
4. Upgrade cluster version from AKS/EKS portal.
5. For EKS, update the add-ons by going to the add-on section (new defaults which are shown)
   1. If any problem occurs during the add-on upgrade, use the following command if there is no customization made as part of the add-on upgrade:
      aws eks update-addon \
          --cluster-name <cluster_name> \
          --addon-name <addon_name> \
          --addon-version <addon_version> \
       --resolve-conflicts OVERWRITE
6. Upgrade all node pools version. This can be done in parallel with a console.
7. Repeat step 4 to step 6 if you want to do multiple cluster version upgrades.
8. Once the cluster, the add-on and the node pool are upgraded to the required version, perform the following:
   1. For NetBackup version 10.5 and above: run the following command to restart:
      ./cloudscale_restart.sh start
   2. For NetBackup version lower than 10.5:
      kubectl patch environment <env_name> -n <namespace> --type=merge -p='{"spec": {"primary": {"paused": false}}}'
9. Resume NetBackup job scheduling.