After installing SURE Firmware tool on Access, UpgradeStatus shows that the rpm installation is 50% completed.

Article: 100053296
Last Published: 2022-06-16
Ratings: 0 0
Product(s): Access, Appliances

Problem

 

After installing the SURE Appliance Firmware tool RPM via the Access Appliance CLISH, the installation shows as 50% completed.  The SURE rpm was installed successfully but the "UpgradeStatus" output indicates that it's still in progress.
 

Error Message

 

va743a-01.Software> UpgradeStatus

The target version is: 7.4.3.200
Current upgrade status: RUNNING. The upgrade is 50% completed.
Latest operations:
 -[2021-12-08 10:00:57] [INFO] V-409-777-1517: Upgrade completed successfully.
 -[2022-06-15 08:18:35] [INFO] {u'node': u'va743a-01', u'umi_code': u'', u'script_status_code': u'', u'msg': u"Product version found in supported version list !!!!\nThis EEB can be installed.\nSuccessfully cleared previous SURE grub menuentry.\nSuccessfully added new SURE grub menuentry.\nRun 'Reboot/Restart' to complete this EEB\n", u'script_detail_output': u''}
 -[2022-06-15 08:19:51] [INFO] {u'node': u'va743a-02', u'umi_code': u'', u'script_detail_output': u'', u'script_status_code': u'', u'msg': u"Product version found in supported version list !!!!\nThis EEB can be installed.\nSuccessfully cleared previous SURE grub menuentry.\nSuccessfully added new SURE grub menuentry.\nRun 'Reboot/Restart' to complete this EEB\n"}

 

This message will appear before and after rebooting, despite the RPM being installed:

# rpm -qa |grep SURE
VRTS_SURE-1.1.1-23.x86_64

 

Cause

 

The SURE Firmware tool was developed for all Veritas appliances, including Netbackup Appliances, which have a slightly different installer process.  While the SURE rpm installs successfully, the status file that provides the output to "UpgradeStatus" is modified incorrectly and leaves the partial progress instead of showing a successful installation.

 

Solution

 

To resolve this, we can edit the file that provides the information to the "UpgradeStatus" command to remove the invalid state.

1. From the Appliance CLISH interface, elevate to a maintenance (bash) prompt:

> Support

> Maintenance

!> Elevate

 

2. Navigate to the following directory: /log/upgrade/upgrading

 

3. Find the timestamped directory that has the correct cluster.info file.

 

va743a-01:/log/upgrade/upgrading # ll
total 16
drwxr-xr-x 2 root root 4096 Jun 15 08:19 20211208080423
drwxr-xr-x 2 root root 4096 Jun 15 08:19 20220615081312
-rw------- 1 root root  243 Jun 15 08:13 eeb_internal_conf.json
-rw-r--r-- 1 root root    0 Dec  8  2021 root_known_hosts
-rw------- 1 root root 2620 Dec  8  2021 upgrade_internal_conf.json


va743a-01:/log/upgrade/upgrading # ll *
-rw------- 1 root root  243 Jun 15 08:13 eeb_internal_conf.json
-rw-r--r-- 1 root root    0 Dec  8  2021 root_known_hosts
-rw------- 1 root root 2620 Dec  8  2021 upgrade_internal_conf.json

20211208080423:
total 52
-rw-r--r-- 1 root root  6779 Jun 15 08:19 cluster.info
-rw-r--r-- 1 root root 25627 Dec  8  2021 post_vol_process.log
-rw-r--r-- 1 root root  5300 Dec  8  2021 va743a-01.info
-rw-r--r-- 1 root root  5997 Dec  8  2021 va743a-02.info

20220615081312:
total 4
-rw-r--r-- 1 root root 1978 Jun 15 08:19 cluster.info

 

Both of the cluster.info files within the datestamp directories will include information about the SURE EEB - we're going to remove the details in the cluster.info file that also includes the previous successful upgrade.  It should be in the subdirectory that also includes the .info files from the two individual nodes' hostnames, as well as the post_vol_process.log.

The files are .json format, so to review the details in a readable format, we can print the contents using python:

# python -m json.tool 20211208080423/cluster.info

<snip>

    {
        "completedPercent": 100,
        "detailedInformation": "V-409-777-1517: Upgrade completed successfully.",
        "loglevel": "info",
        "patchName": "VRTSaccess-app-update-7.4.3.200-1.x86_64.rpm",
        "processName": "Upgrade_msdp",
        "processState": "COMPLETED",
        "time": "20211208100057"
    },
    {
        "completedPercent": "50",
        "detailedInformation": {
            "msg": "Product version found in supported version list !!!!\nThis EEB can be installed.\nSuccessfully cleared previous SURE grub menuentry.\nSuccessfully added new SURE grub menuentry.\nRun 'Reboot/Restart' to complete this EEB\n",
            "node": "va743a-01",
            "script_detail_output": "",
            "script_status_code": "",
            "umi_code": ""
        },

</snip>

 

The entry following "Upgrade completed successfully" starts the SURE EEB installation messages.

Printing the raw file (with "cat") without using python shows the details on one continuous line like this:


<snip>
{"detailedInformation": "V-409-777-1517: Upgrade completed successfully.", "completedPercent": 100, "patchName": "VRTSaccess-app-update-7.4.3.200-1.x86_64.rpm", "loglevel": "info", "processName": "Upgrade_msdp", "processState": "COMPLETED", "time": "20211208100057"}, {"detailedInformation": {"node": "va743a-01", "umi_code": "", "script_status_code": "", "msg": "Product version found in supported version list !!!!\nThis EEB can be installed.\nSuccessfully cleared previous SURE grub menuentry.\nSuccessfully added new SURE grub menuentry.\nRun 'Reboot/Restart' to complete this EEB\n", "script_detail_output": ""},
</snip>

 

We want to remove everything following the { } section with the previous successful upgrade, but make sure to include the close square bracket "]" at the end of the file.  

 

4. Backup the file as a precaution, in cause the manual edits do not succeed:


# cp cluster.info cluster.info_backup

 

5. In vi/vim, delete everything from the cursor to the end with d$:

...
{"detailedInformation": "V-409-777-1517: Upgrade completed successfully.", "completedPercent": 100, "patchName": "VRTSaccess-app-update-7.4.3.200-1.x86_64.rpm", "loglevel": "info", "processName": "Upgrade_msdp", "processState": "COMPLETED", "time": "20211208100057"} <<DELETE FROM HERE TO END OF LINE>> 

Add the ] bracket at the end.


{"detailedInformation": "V-409-777-1517: Upgrade completed successfully.", "completedPercent": 100, "patchName": "VRTSaccess-app-update-7.4.3.200-1.x86_64.rpm", "loglevel": "info", "processName": "Upgrade_msdp", "processState": "COMPLETED", "time": "20211208100057"}]

 

6. Validate that the .json file is readable after editing, to check for syntax errors (make sure the file ends with the ] bracket):

<snip>
    {
        "completedPercent": 100,
        "detailedInformation": "V-409-777-1517: Upgrade completed successfully.",
        "loglevel": "info",
        "patchName": "VRTSaccess-app-update-7.4.3.200-1.x86_64.rpm",
        "processName": "Upgrade_msdp",
        "processState": "COMPLETED",
        "time": "20211208100057"
    }
]

</snip, end of file>

 

7. Validate the new contents of the file are readable by the following script - note that the more recent entries are at the top of the output, compared to the python output:
 

# /opt/veritas/appliance/softwaremgmt/scripts/get_upgrade_state.py --mode overall
 

{"msg": "Successfully read status file cluster.info.\n", "result": true, "details": {"Status": "COMPLETED", "Percent": 100, "Log": [{"Loglevel": "info", "Message": "V-409-777-1517: Upgrade completed successfully.", "Time": "20211208100057"}, ...

 

8. Return to the Appliance CLISH and check the UpgradeStatus again:

 

va743a-01.Software> UpgradeStatus

The target version is: 7.4.3.200
Current upgrade status: COMPLETED. The upgrade is 100% completed.
Latest operations:
 -[2021-12-08 09:34:44] [INFO] V-409-777-30006: Upgrading the Veritas Data Deduplication packages...
 -[2021-12-08 09:56:25] [INFO] Running selftest.
 -[2021-12-08 10:00:57] [INFO] V-409-777-1517: Upgrade completed successfully.


9. Repeat the steps on the partner node.

 

 

 

Was this content helpful?