Search <book_title>...

Storage Foundation for Oracle® RAC 7.3.1 Administrator's Guide - Linux

Last Published: 2018-01-16

Product(s): InfoScale & Storage Foundation (7.3.1)

Section I. SF Oracle RAC concepts and administration
1. Overview of Storage Foundation for Oracle RAC
2. Administering SF Oracle RAC and its components
Section II. Performance and troubleshooting
Section III. Reference
1. Appendix A. List of SF Oracle RAC health checks
2. Appendix B. Error messages

Removing preexisting keys

If you encountered a split-brain condition, use the vxfenclearpre utility to remove CP Servers, SCSI-3 registrations, and reservations on the coordinator disks, Coordination Point servers, as well as on the data disks in all shared disk groups.

You can also use this procedure to remove the registration and reservation keys of another node or other nodes on shared disks or CP server.

To clear keys after split-brain

Stop VCS on all nodes.
```
# hastop -all
```
Make sure that the port h is closed on all the nodes. Run the following command on each node to verify that the port h is closed:
```
# gabconfig -a
```
Port h must not appear in the output.
Stop I/O fencing on all nodes. Enter the following command on each node:
For RHEL 7, SLES 12, and supported RHEL distributions:
# systemctl stop vxfen
For earlier versions of RHEL, SLES, and supported RHEL distributions:
# /etc/init.d/vxfen stop
If you have any applications that run outside of VCS control that have access to the shared storage, then shut down all other nodes in the cluster that have access to the shared storage. This prevents data corruption.
Start the vxfenclearpre script:
```
# /opt/VRTSvcs/vxfen/bin/vxfenclearpre
```

Read the script's introduction and warning. Then, you can choose to let the script run.

Do you still want to continue: [y/n] (default : n) y

In some cases, informational messages resembling the following may appear on the console of one of the nodes in the cluster when a node is ejected from a disk/LUN. You can ignore these informational messages.

<date> <system name> scsi: WARNING: /sbus@3,0/lpfs@0,0/
sd@0,1(sd91):
<date> <system name> Error for Command: <undecoded 
cmd 0x5f> Error Level: Informational
<date> <system name> scsi: Requested Block: 0 Error Block 0
<date> <system name> scsi: Vendor: <vendor> Serial Number: 
0400759B006E
<date> <system name> scsi: Sense Key: Unit Attention
<date> <system name> scsi: ASC: 0x2a (<vendor unique code 
0x2a>), ASCQ: 0x4, FRU: 0x0

The script cleans up the disks and displays the following status messages.

Cleaning up the coordinator disks...

Cleared keys from n out of n disks, 
where n is the total number of disks.

Successfully removed SCSI-3 persistent registrations
from the coordinator disks.

Cleaning up the Coordination Point Servers...

...................
[10.209.80.194]:50001: Cleared all registrations
[10.209.75.118]:443: Cleared all registrations

Successfully removed registrations from the Coordination Point Servers.

Cleaning up the data disks for all shared disk groups ...

Successfully removed SCSI-3 persistent registration and
reservations from the shared data disks.

See the log file /var/VRTSvcs/log/vxfen/vxfen.log

You can retry starting fencing module. In order to restart the whole 
product, you might want to reboot the system.

Start the fencing module on all the nodes.
For RHEL 7, SLES 12, and supported RHEL distributions:
# systemctl start vxfen
For earlier versions of RHEL, SLES, and supported RHEL distributions:
# /etc/init.d/vxfen start
Start VCS on all nodes.
```
# hastart
```