Problem
drcontrol --recover_last_cloud_catalyst_image may finish with Status 1.
Error Message
Depending on the recovery environment, errors occur in the following step.
Please refer to the following steps in DisasterRecoveryforNetBackup™CloudCatalyst under the topic "Recovering the masterserver(partial) and the MSDP cloud storage server when NetBackup Cloud Catalyst is enabled"
- Step 8. Recover the MSDP cloud storage server files and the FSDB database. Run the following command:
# ./drcontrol --recover_last_cloud_catalyst_image --policy msdp_catalog_catalyst
WARNING! this operation will use bprestore to restore the following paths
"/MSDP/cache/etc"
"/MSDP/cache/fsdb"
"/MSDP/cache/userdata/proc/cloud.lsu"
:
ERROR: /usr/openv/esfs/bin/cred_ioctl exited with error 1:
Usage: /usr/openv/esfs/bin/cred_ioctl [-update] <mount point> <key> <secretkey>
Cause
CSSC_USERNAME and CSSC_PASSWORD are undefined by default.
Solution
This procedure assumes the following:
- Latest MSDP catalog backup exists.
- If swapping in a new host, the host name is the same as the name of the previous host.
- Install NetBackup on the master server
- Install NetBackup on the MSDP cloud storage server
- On the MSDP cloud storage server, edit the esfs_init.sh file, located in the following directory:
/usr/openv/esfs/scripts/esfs_init.sh
Add the readonly option to the MKESFS call. This assures that the data can be read for restores, but that no new backups are written to the media server. The option also assures that no existing backup images are expired or deleted.
For example:
Before the change:
MKESFS_OUTPUT=$( ${MKESFS} -o cache_dir="${CACHEDIR_AS_FUSEOPT}",storage_server=${CLDSRV},fs_name=${FSNAME},media_server=${MEDIA},master_server=${MASTER},fs_version=${FSVERSION},cld_db_path=$ {CLDDBPATH}${READONLY}${DR_IN_CLOUD}${LOG_LEVEL} 2>&1 )
After the change:
MKESFS_OUTPUT=$( ${MKESFS} -o readonly,cache_dir="${CACHEDIR_AS_FUSEOPT}",storage_server=${CLDSRV},fs_name=${FSNAME},media_server=${MEDIA},master_server=${MASTER},fs_version=${FSVERSION},cld_db_path=$ {CLDDBPATH}${READONLY}${DR_IN_CLOUD}${LOG_LEVEL} 2>&1 )
# Entry at line 197.
Notes on the read only option:
If future duplication or backup jobs try to write to this media server, the job fails with status code 84 (media write error), because the storage is set to readonly.
The readonly option can be removed at a later date to allow the MSDP cloud storage server to once again be used for duplication or backup jobs. However, the best approach is to create a new MSDP cloud storage server for this purpose and to keep this recovered server in a readonly mode until the data is no longer needed and it can be retired.
- In the NetBackup Administration Console on the master server, navigate to the Catalog utility. Import the images that were created by the MSDP catalog policy from the storage unit where they were saved. (That is, the MSDP catalog backup policy that was originally created using the drcontrol --new_policy command.)
- Configure the MSDP cloud storage server.
- If the MSDP cloud storage server is a non-appliance media server:
- In the NetBackup Administration Console, click Configure Cloud Storage Servers to launch the Cloud Storage Server Configuration Wizard.
- Be sure to enable the option Enable NetBackup CloudCatalyst.
- If the MSDP cloud storage server is a CloudCatalyst Appliance:
- Perform the initial configuration on the appliance.
- From the main shell menu, launch the Appliance Cloud Storage Server Configuration Wizard.
- Set the USERNAME and PASSWORD variables in your environment:
- export CSSC_USERNAME=<CLOUD_USERNAME>
- export CSSC_PASSWORD=<CLOUD_PASSWORD>
Note:In case of IAM Role or CREDS_CAPS credential broker type, provide the credentials you had entered when configuring the CloudCatalyst storage eserver.
For more information, review the Managing CloudCatalyst storage server with IAM Role or CREDS_CAPS credential broker type section in the NetBackup Deduplication Guide.
- On the MSDP cloud storage server, use the drcontrol command to create an MSDP catalog policy if such a policy does not already exist.
Run the following command. For catalog_policy_name, use the same name as the original catalog policy:
/usr/openv/pdde/pdcr/bin/drcontrol --new_policy --policy <CATALOG_POLICY_NAME>
- On the MSDP cloud storage server, run the following command:
/usr/openv/pdde/pdcr/bin/drcontrol --initialize_DR --policy <CATALOG_POLICY_NAME>
- Recover the MSDP cloud storage server files and the FSDB database. Run the following command:
/usr/openv/pdde/pdcr/bin/drcontrol --recover_last_cloud_catalyst_image --policy <CATALOG_POLICY_NAME>
- Use the Backup, Archive, and Restore client interface to restore the files and directories protected by the MSDP catalog policy (MSDP_policy_name in the previous steps), except for the cache directory.
That is, select /cache_directory/storage and /usr for restore, but do not select /cache_directory/cache for restore.
- Start spad on the MSDP cloud storage server by running the following command:
/usr/openv/pdde/pdconfigure/pdde spad start
- Recover the MSDP catalog from the catalog shadow files by running the following command on the MSDP cloud storage server:
/usr/openv/pdde/pdcr/bin/cacontrol --catalog disaster_recovery
- Start spoold on the MSDP cloud storage server by running the following command:
/usr/openv/pdde/pdconfigure/pdde spoold start
- In the NetBackup Administration Console on the master server, navigate to the Catalog utility. Import the images from the MSDP cloud storage server to make the images available for restore.
- Use the Catalog utility to run a Verify operation on an existing backup image. The Verify operation should verify the following:
- That the data has been downloaded from the cloud.
- That the image is available for restore.
- Some time after verifying that the recovery operation is successful, you can reclaim some disk space by removing saved MDSP catalog directories that were created during the recovery process. To do so, run the following command:
/usr/openv/pdde/pdcr/bin/drcontrol --cleanup