VCS DiskGoup agent import fails due to UseFence=SCSI3 and vxdctl scsi3pr turned off by data_disk_fencing=off attribute

Description

The Veritas Cluster Server (VCS) "UseFence=SCSI3" attribute is defined in the /etc/VRTSvcs/conf/config/main.cf file.
The VCS DiskGroup agents may fail to online Veritas Volume Manager (VxVM) DiskGroups when the "vxdctl scsi3pr" setting is turned off in certain situations.

Sample error: /var/VRTSvcs/log/DiskGroup_A.log

Jun xx 00:0x:21 testserver AgentFramework[135242]: VCS ERROR V-16-10031-1562 DiskGroup:datadg:online:Diskgroup [datadg] failed to import with groupreserve option as scsi3pr is disabled.Syntax called by VCS DiskGroup agent:

When attempting to import the VxVM disk group with reservations outside of VCS, it fails due to "vxdctl scsi3pr" setting being turned off.
The import will not work with SCSI-3 reservations until the "vxdctl scsi3pr" setting is turned on.

Example

# /usr/sbin/vxdg -o groupreserve=VCS -t import datadg
VxVM vxdg ERROR V-5-1-19179 Disk group datadg: import failed:
SCSI3-PR is disabled. Enable it to allow operation

Table 1.0
 

By design, SCSI3 reservations are disabled by scsi3pr functionality at system startup when the /etc/vxenviron file is defined as follows:

data_disk_fencing=off


Example

# cat /etc/vxenviron
data_disk_fencing=off


The system startup file "/sbin/vxfen-startup" checks the contents of the /etc/vxenviron file, and if "data_disk_fencing=off", the VxVM vxdctl scsi3pr functionality is turned off.


# grep -i scsi3pr /sbin/vxfen-startup
                        vm_setting=`${VXDCTL} scsi3pr | ${SED} 's/.*: //'`
                                ${VXDCTL} scsi3pr off

File: /sbin/vxfen-startup

#
#       Non-SCSI3 environment fencing
#       If non-scsi3 fencing has to be configured, disable scsi3 pr done by VM.
#
if [ -f /etc/vxenviron ]; then
        # Source the file to get the value of data_disk_fencing
        . /etc/vxenviron

        if [ "${data_disk_fencing}" = "off" ]; then
                # data disk fencing is off

                # set LLT up to panic if timers are stuck. Needed since
                # we do not have SCSI3 PR protection
                /sbin/lltconfig -p on

                # ensure that VM doesn't use SCSI3 PR
                if [ -x ${VXDCTL} ]; then
                        vm_setting=`${VXDCTL} scsi3pr | ${SED} 's/.*: //'`
                        if [ "${vm_setting}" = "on" ]; then
                                ${VXDCTL} scsi3pr off
                        fi
                fi
        fi
fi

NOTE: Define "data_disk_fencing=off" when CP servers are used in place of traditional SCSI-3 coordinator disk-based devices.

If CP servers are deployed, the data_disk_fencing must be set to "off" as per the table above. 

If the "/etc/vxenviron" file does not exist, the VxVM "vxdctl scsi3pr" setting will handle SCSI3-PR functionality and it will be enabled by default



Solution

For SCSI-3 fencing disks

1.] Update the /etc/vxenviron file and set "data_disk_fencing=on"

# cat /etc/vxenviron
data_disk_fencing=on

2.] Enable scsi3pr

# vxdctl scsi3pr on

3.] Verify scsi3pr has been enabled at the VxVM level:

# vxdctl scsi3pr
scsi3pr: on

4.] Now clear any resource faults and online the VCS ServiceGroup and the DiskGroup resource will now come online.

# hastatus -sum

# hares -clear <resource-name>

# hagrp -online <Service-Group> -sys <system-name> 

# hastatus -sum


For CPS environments

1.] Update the /etc/vxenviron file and set "data_disk_fencing=off"

# cat /etc/vxenviron
data_disk_fencing=off

Optional, following the next reboot, the following will be automated during the system startup:

2.] Disable scsi3pr

# vxdctl scsi3pr off

3.] Verify scsi3pr has been disabled at the VxVM level:

# vxdctl scsi3pr
scsi3pr: off

 

VCS DiskGroup Agent: Reservation Attribute
 

The "Reservation" attribute determines if you want to enable SCSI-3 reservation.

This attribute can have one of the following three values:

■ ClusterDefault—The disk group is imported with SCSI-3 reservation if the value of the cluster-level UseFence attribute is
SCSI3. If the value of the cluster-level UseFence attribute is NONE, the disk group is imported without reservation.

■ SCSI3—The disk group is imported with SCSI-3 reservation if the value of the cluster-level UseFence attribute is SCSI3.

■ NONE—The disk group is imported without SCSI-3 reservation.

To import a disk group with SCSI-3 reservations, ensure that the disks of the disk group are SCSI-3 persistent reservation (PR) compliant.

Type and dimension: string-scalar
Default: ClusterDefault
Example: "SCSI3"