Problem
The system crashes during an upgrade. The crash can happen on either of the two compute nodes. Node A is the node where the command Manage > Software > Install
is executed. Node B is the other node in the system.
When the system crashes, the core file is generated under /opt/VRTSnas/core/kernel
on the node that crashed.
Error Message
- Node A crashes
The upgrade process is interrupted and Node A reboots automatically. The upgrade status on Node A displays the following messages:
- [Info] Completed the following pre-upgrade tasks on the node Node_B. The appliance upgrade is 20% complete.
- Created the pre-upgrade checkpoint
- Added the service for the post-upgrade tasks
- [Info] Starting upgrade phase one on the current node Node_A...
The last few lines in the upgrade log on Node A should contain the message:
"upgrade.upgrade_comm.py - DEBUG - "Command is:./access_ctrl.sh stop"
or
"upgrade.upgrade_comm.pyc - DEBUG - Command is:./access_ctrl.sh start"
- Node B crashes
The upgrade process is interrupted and node B reboots automatically. The upgrade status on Node A displays the following messages:
- [Info] Upgrade phase one is completed on the current node Node_A. The appliance upgrade is 35% complete.
- Upgraded the Access software
- Upgraded the appliance software
- Upgraded the VRTS kernel packages
- [Info] Starting upgrading the node Node_B…"
The last few lines in the upgrade log on Node A should contain the message:
"upgrade.upgrade_comm.py - DEBUG - Command is:ssh root@Node_B 'cd /inst/patch/appliance/installed/7.4.*/scripts;./access_ctrl.sh stop
'"
Cause
When the Veritas Virtual Manager (VxVM) service is stopped, the vxio driver is unloaded, which results in the system crash.
Solution
Node A crashes
- Log on to Node A. Elevate to the command shell.
- Remove the upgrade state file and lock files on Node A:
-
rm /inst/patch/appliance/upgrading/state.info
-
rm /inst/patch/appliance/upgrading/*.lock
-
-
Roll back both the nodes manually. For details refer to 100044228
-
Upgrade the node again by using the
Manage > Software > Install
command in the Access Appliance shell menu.
Node B crashes