Storage Foundation for Oracle® RAC 7.4.1 Administrator's Guide - Linux

To replace a disk in a coordinator disk group when the cluster is online

1. Make sure system-to-system communication is functioning properly.
2. Determine the value of the FaultTolerance attribute.

   # hares -display coordpoint -attribute FaultTolerance -localclus

3. Estimate the number of coordination points you plan to use as part of the fencing configuration.
4. Set the value of the FaultTolerance attribute to 0.

   Note:

   It is necessary to set the value to 0 because later in the procedure you need to reset the value of this attribute to a value that is lower than the number of coordination points. This ensures that the Coordpoint Agent does not fault.

5. Check the existing value of the LevelTwoMonitorFreq attribute.

   #hares -display coordpoint -attribute LevelTwoMonitorFreq -localclus

   Note:

   Make a note of the attribute value before you proceed to the next step. After migration, when you re-enable the attribute you want to set it to the same value.

   You can also run the hares -display coordpoint to find out whether the LevelTwoMonitorFreq value is set.

6. Disable level two monitoring of CoordPoint agent.

   # hares -modify coordpoint LevelTwoMonitorFreq 0

7. Make sure that the cluster is online.

   # vxfenadm -d

   I/O Fencing Cluster Information:
   ================================
   Fencing Protocol Version: 201
   Fencing Mode: SCSI3
   Fencing SCSI3 Disk Policy: dmp
   Cluster Members:
   		* 0 (sys1)
   		1 (sys2)
   RFSM State Information:
   		node 0 in state 8 (running)
   		node 1 in state 8 (running)

8. Import the coordinator disk group.

   The file /etc/vxfendg includes the name of the disk group (typically, vxfencoorddg) that contains the coordinator disks, so use the command:

   # vxdg -tfC import 'cat /etc/vxfendg'

   where:

   -t specifies that the disk group is imported only until the node restarts.

   -f specifies that the import is to be done forcibly, which is necessary if one or more disks is not accessible.

   -C specifies that any import locks are removed.

9. If your setup uses VRTSvxvm version, then skip to step 10. You need not set coordinator=off to add or remove disks. For other VxVM versions, perform this step:

   Where version is the specific release version.

   Turn off the coordinator attribute value for the coordinator disk group.

   # vxdg -g vxfencoorddg set coordinator=off

10. To remove disks from the coordinator disk group, use the VxVM disk administrator utility vxdiskadm.

11. Perform the following steps to add new disks to the coordinator disk group:

    • Add new disks to the node.

    • Initialize the new disks as VxVM disks.

    • Check the disks for I/O fencing compliance.

    • Add the new disks to the coordinator disk group and set the coordinator attribute value as "on" for the coordinator disk group.

    See the Storage Foundation for Oracle RAC Configuration and Upgrade Guide for detailed instructions.

    Note that though the disk group content changes, the I/O fencing remains in the same state.

12. From one node, start the vxfenswap utility. You must specify the disk group to the utility.

    The utility performs the following tasks:

    • Backs up the existing /etc/vxfentab file.

    • Creates a test file /etc/vxfentab.test for the disk group that is modified on each node.

    • Reads the disk group you specified in the vxfenswap command and adds the disk group to the /etc/vxfentab.test file on each node.

    • Verifies that the serial number of the new disks are identical on all the nodes. The script terminates if the check fails.

    • Verifies that the new disks can support I/O fencing on each node.

13. If the disk verification passes, the utility reports success and asks if you want to commit the new set of coordinator disks.

14. Confirm whether you want to clear the keys on the coordination points and proceed with the vxfenswap operation.

    Do you want to clear the keys on the coordination points 
    and proceed with the vxfenswap operation? [y/n] (default: n) y

15. Review the message that the utility displays and confirm that you want to commit the new set of coordinator disks. Else skip to step 16.

    Do you wish to commit this change? [y/n] (default: n) y

    If the utility successfully commits, the utility moves the /etc/vxfentab.test file to the /etc/vxfentab file.

16. If you do not want to commit the new set of coordinator disks, answer n.

    The vxfenswap utility rolls back the disk replacement operation.

17. If coordinator flag was set to off in step 9, then set it on.

    # vxdg -g vxfencoorddg set coordinator=on

18. Deport the diskgroup.

    # vxdg deport vxfencoorddg 

19. Re-enable the LevelTwoMonitorFreq attribute of the CoordPoint agent.You may want to use the value that was set before disabling the attribute.

    # hares -modify coordpoint LevelTwoMonitorFreq Frequencyvalue

    where Frequencyvalue is the value of the attribute.

20. Set the FaultTolerance attribute to a value that is lower than 50% of the total number of coordination points.

    For example, if there are four (4) coordination points in your configuration, then the attribute value must be lower than two (2).If you set it to a higher value than two (2) the CoordPoint agent faults.