Unable to start Veritas Cluster Server I/O Fencing when booting or using the appropriate start command.
Problem
During boot or when using the appropriate start command, I/O fencing does not start due to a setting in the configuration file /etc/default/vxfen . Separate configuration files for 4 other VCS services exist in the same directory; see information below for more detail.
This /etc/default/vxfen file, implemented in version 5.1, contains settings which may be used in troubleshooting scenarios to enable and disable the start and stopping of the I/O fencing service. While this article will show how this can be manually modified, be aware that if this file is configured to not allow the service to start, it may be an indication that the service needs to be configured ('installsfha -configure' for example). This article is presented using /etc/default/vxfen however this directory contains files for the cluster services started by init; amf, llt, gab, vxfen and vcs.
OS locations for these files:
Solaris: /etc/default
Linux: /etc/sysconfig
HP-UX: /etc/default
AIX: /etc/default
Error Message
from /var/adm/messages:
Oct 13 09:11:19 maztoy svc.startd[7]: system/vxfen:default failed: transitioned to maintenance (see 'svcs -xv' for details)
-----------------------------------------------------
# svcs -xv
...
svc:/system/vxfen:default (Veritas I/O Fencing (VXFEN) Init service)
State: maintenance since October 13, 2011 09:11:19 AM PDT
Reason: Start method failed repeatedly, last exited with status 2.
See: http://sun.com/msg/SMF-8000-KS
See: man -M /opt/VRTS/man/man1m/ -s 1M vxfenconfig
See: /var/svc/log/system-vxfen:default.log
Impact: This service is not running.
-----------------------------------------------------
# cat /var/svc/log/system-vxfen:default.log
...
[ Oct 13 09:11:19 Executing start method ("/lib/svc/method/vxfen start") ]
This script is not allowed to start vxfen. VXFEN_START is not 1
[ Oct 13 09:11:19 Method "start" exited with status 2 ]
[ Oct 13 09:11:19 Executing start method ("/lib/svc/method/vxfen start") ]
This script is not allowed to start vxfen. VXFEN_START is not 1
[ Oct 13 09:11:19 Method "start" exited with status 2 ]
Cause
Settings in /etc/default/gab are set to not allow the start and/or stopping of I/O Fencing.
# cd /etc/default
# cat vxfen
#
# This file is sourced :
# from /etc/init.d/vxfen for Solaris < 2.10
# from /lib/svc/method/vxfen for Solaris 2.10
#
# Set the two environment variables below as follows:
#
# 1 = start or stop vxfen
# 0 = do not start or stop vxfen
#
VXFEN_START=0
VXFEN_STOP=0
Solution
Edit this file to turn on start and stop capability then start the service.
1) Edit the file /etc/default/vxfen to set the start and stop values to 1
# cat vxfen
#
# This file is sourced :
# from /etc/init.d/vxfen for Solaris < 2.10
# from /lib/svc/method/vxfen for Solaris 2.10
#
# Set the two environment variables below as follows:
#
# 1 = start or stop vxfen
# 0 = do not start or stop vxfen
#
VXFEN_START=1
VXFEN_STOP=1
2) Verify the status of the service
# gabconfig -a
GAB Port Memberships
===============================================================
Port a gen 357717 membership 01
<=== port b would be seen here
3) Verify the status of the service
# svcs -a|grep vxfen
maintenance 9:11:19 svc:/system/vxfen:default
4) Restart the service
# svcadm restart vxfen
5) Validate the status of the service
# svcs -a|grep vxfen
maintenance 9:11:19 svc:/system/vxfen:default
6) Check the Operating Systems (OS's) services log
# svcs -xv
...[ Oct 13 09:11:19 Executing start method ("/lib/svc/method/vxfen start") ]
This script is not allowed to start vxfen. VXFEN_START is not 1
[ Oct 13 09:11:19 Method "start" exited with status 2 ]
from /var/adm/messages:
Oct 13 09:11:19 maztoy svc.startd[7]: system/vxfen:default failed: transitioned to maintenance (see 'svcs -xv' for details)
Oct 13 09:18:07 maztoy vxfen: NOTICE: VXFEN INFO V-11-1-VxFEN configured at protocol
[ Oct 13 09:11:19 Executing start method ("/lib/svc/method/vxfen start") ]
This script is not allowed to start vxfen. VXFEN_START is not 1
[ Oct 13 09:11:19 Method "start" exited with status 2 ]
7) Restarting the OS service did not seem to re-read the configuration file; a stop and start of the service will be attempted.
# svcadm disable vxfen
# svcadm enable vxfen
8) Checking the state of the service
# svcs -a|grep vxfen
online 9:18:07 svc:/system/vxfen:default
9) The OS is reporting that the service is online; checking /var/adm/messages for log entries created by the Fencing module
Oct 13 09:18:07 maztoy vxfen: NOTICE: VXFEN INFO V-11-1-VxFEN configured at protocol version 20
Oct 13 09:18:13 maztoy gab: GAB INFO V-15-1-20036 Port b gen 357716 membership 01
Oct 13 09:18:13 maztoy vxfen: NOTICE: VXFEN INFO V-11-1-35 Fencing driver going into RUNNING state
10) Verifying that the Fencing port is registered (by GAB)
# gabconfig -a
GAB Port Memberships
===============================================================
Port a gen 357717 membership 01
Port b gen 357716 membership 01 <=== indicates that I/O fencing has been
configured and started