VxVM: Newly added device paths & controllers are not seen (visible) as they are marked as EXCLUDED due to array_type mismatch ALUA vs A/A

VxVM: Newly added device paths & controllers are not seen (visible) as they are marked as EXCLUDED due to array_type mismatch ALUA vs A/A

Article: 100047777
Last Published: 2020-05-28
Ratings: 13 0
Product(s): InfoScale & Storage Foundation

Problem


Additional device paths via new controllers are NOT seen by Veritas Volume Manager (VxVM) for a HUAWEI XSG1 storage array (enclosure).

The storage array is already connected via four existing HBAs: c7, c8, c12, c19. The objective was to add additional paths via new controllers: c9 and c20.

The system has already been rebooted, however, VxVM still does not display any c9 or c20 related paths. The OS can see these paths and they are accessible.

# vxdmpadm listctlr all output
CTLR_NAME       ENCLR_TYPE      STATE        ENCLR_NAME      PATH_COUNT
=========================================================================
c12             HUAWEI-XSG1     ENABLED      huawei-xsg10         64
c19             HUAWEI-XSG1     ENABLED      huawei-xsg10         64
c7              HUAWEI-XSG1     ENABLED      huawei-xsg10         512
c8              HUAWEI-XSG1     ENABLED      huawei-xsg10         512

Error Message


When reviewing the evidence collected, the VxVM command "vxddladm list devices" shows the c9 and c20 paths as EXCLUDED.


# grep ^c20 vxddladm_list_devices | head
c20t26123C9D56E17026d1 c20_p9_t0    Online  EXCLUDED
c20t26123C9D56E17026d2 c20_p9_t0    Online  EXCLUDED
c20t26123C9D56E17026d3 c20_p9_t0    Online  EXCLUDED
c20t26123C9D56E17026d4 c20_p9_t0    Online  EXCLUDED
c20t26123C9D56E17026d5 c20_p9_t0    Online  EXCLUDED
c20t26123C9D56E17026d6 c20_p9_t0    Online  EXCLUDED
c20t26123C9D56E17026d7 c20_p9_t0    Online  EXCLUDED
c20t26123C9D56E17026d8 c20_p9_t0    Online  EXCLUDED
c20t26123C9D56E17026d9 c20_p9_t0    Online  EXCLUDED
c20t26123C9D56E17026d10 c20_p9_t0    Online  EXCLUDED


# grep ^c9 vxddladm_list_devices | head
c9t24001C20DBEE054Ed1 c9_p9_t0     Online  EXCLUDED
c9t24001C20DBEE054Ed2 c9_p9_t0     Online  EXCLUDED
c9t24001C20DBEE054Ed3 c9_p9_t0     Online  EXCLUDED
c9t24001C20DBEE054Ed4 c9_p9_t0     Online  EXCLUDED
c9t24001C20DBEE054Ed5 c9_p9_t0     Online  EXCLUDED
c9t24001C20DBEE054Ed6 c9_p9_t0     Online  EXCLUDED
c9t24001C20DBEE054Ed7 c9_p9_t0     Online  EXCLUDED
c9t24001C20DBEE054Ed8 c9_p9_t0     Online  EXCLUDED
c9t24001C20DBEE054Ed9 c9_p9_t0     Online  EXCLUDED
c9t24001C20DBEE054Ed10 c9_p9_t0     Online  EXCLUDED
 

Cause


Veritas DMP cannot handle a mix of array_types presented from different controllers connected to the same physical enclosure, i.e. ALUA, A/P, A/A and so on. 


To troubleshoot the issue, place the vxconfigd daemon into level 9 debug mode as shown below:

# vxdctl debug 9 /var/tmp/vxconfigd_debug9.log
 

# vxdctl debug get
loglevel: 9 taggedlevel: 0 logfile: /var/tmp/vxconfigd_debug9.log

 

# date; vxdisk scandisks

To turn off debugging for vxconfigd, type:
 

# vxdctl debug 0
 

---------------------------------------------------------------------------------------------------------------------------------------------------
Log Review: vxconfigd
---------------------------------------------------------------------------------------------------------------------------------------------------
 

From the collected vxconfigd debug log, both the c9 & c20 related paths are detected by VxVM & DMP:
 

# grep devno vxconfigd_debug9.log | grep -w dev | grep c9 | head
05/26 11:18:16:  VxVM vxconfigd DEBUG  V-5-1-14097  c9t24001C20DBEE054Ed32 devno 0x3d82d00 dev 246/11520 flags 0x802 type 3
05/26 11:18:16:  VxVM vxconfigd DEBUG  V-5-1-14097  c9t24001C20DBEE054Ed64 devno 0x3d82c00 dev 246/11264 flags 0x802 type 3
05/26 11:18:16:  VxVM vxconfigd DEBUG  V-5-1-14097  c9t2410E000844DB59Ed32 devno 0x3d82b00 dev 246/11008 flags 0x802 type 3
05/26 11:18:16:  VxVM vxconfigd DEBUG  V-5-1-14097  c9t2410E000844DB59Ed64 devno 0x3d82a00 dev 246/10752 flags 0x802 type 3
05/26 11:18:16:  VxVM vxconfigd DEBUG  V-5-1-14097  c9t24101C20DBEE054Ed32 devno 0x3d82500 dev 246/9472 flags 0x802 type 3
05/26 11:18:16:  VxVM vxconfigd DEBUG  V-5-1-14097  c9t24101C20DBEE054Ed64 devno 0x3d82400 dev 246/9216 flags 0x802 type 3
05/26 11:18:16:  VxVM vxconfigd DEBUG  V-5-1-14097  c9t24103C9D56E17026d32 devno 0x3d82900 dev 246/10496 flags 0x802 type 3
05/26 11:18:16:  VxVM vxconfigd DEBUG  V-5-1-14097  c9t24103C9D56E17026d64 devno 0x3d82800 dev 246/10240 flags 0x802 type 3
05/26 11:18:16:  VxVM vxconfigd DEBUG  V-5-1-14097  c9t2400E000844DB5A6d32 devno 0x3d82f00 dev 246/12032 flags 0x802 type 3
05/26 11:18:16:  VxVM vxconfigd DEBUG  V-5-1-14097  c9t2400E000844DB5A6d64 devno 0x3d82e00 dev 246/11776 flags 0x802 type 3


# grep devno vxconfigd_debug9.log| grep -w dev | grep c20 | head
05/26 11:18:16:  VxVM vxconfigd DEBUG  V-5-1-14097  c20t26123C9D56E17026d32 devno 0x3d83900 dev 246/14592 flags 0x802 type 3
05/26 11:18:16:  VxVM vxconfigd DEBUG  V-5-1-14097  c20t26123C9D56E17026d64 devno 0x3d83800 dev 246/14336 flags 0x802 type 3
05/26 11:18:16:  VxVM vxconfigd DEBUG  V-5-1-14097  c20t2612E000844DB59Ed32 devno 0x3d83b00 dev 246/15104 flags 0x802 type 3
05/26 11:18:16:  VxVM vxconfigd DEBUG  V-5-1-14097  c20t2612E000844DB59Ed64 devno 0x3d83a00 dev 246/14848 flags 0x802 type 3
05/26 11:18:16:  VxVM vxconfigd DEBUG  V-5-1-14097  c20t2602E000844DB5A6d32 devno 0x3d83f00 dev 246/16128 flags 0x802 type 3
05/26 11:18:16:  VxVM vxconfigd DEBUG  V-5-1-14097  c20t2602E000844DB5A6d64 devno 0x3d83e00 dev 246/15872 flags 0x802 type 3
05/26 11:18:16:  VxVM vxconfigd DEBUG  V-5-1-14097  c20t26023C9D56E17026d32 devno 0x3d84100 dev 246/16640 flags 0x802 type 3
05/26 11:18:16:  VxVM vxconfigd DEBUG  V-5-1-14097  c20t26023C9D56E17026d64 devno 0x3d84000 dev 246/16384 flags 0x802 type 3
05/26 11:18:16:  VxVM vxconfigd DEBUG  V-5-1-14097  c20t26121C20DBEE054Ed32 devno 0x3d83500 dev 246/13568 flags 0x802 type 3
05/26 11:18:16:  VxVM vxconfigd DEBUG  V-5-1-14097  c20t26121C20DBEE054Ed64 devno 0x3d83400 dev 246/13312 flags 0x802 type 3

 

---------------------------------------------------------------------------------------------------------------------------------------------------

The DMP structures for the c20 controller paths are being discovered:

# grep setup_target_names vxconfigd_debug9.log | grep "controller = 20"
<snippet>
05/26 11:17:09:  VxVM vxconfigd DEBUG  V-5-1-24847 setup_target_names: controller = 20 addr = w2612e000844db5a6,40 flags = 0xe notarg = 26021
05/26 11:17:09:  VxVM vxconfigd DEBUG  V-5-1-24847 setup_target_names: controller = 20 addr = w2612e000844db5a6,3f flags = 0xe notarg = 26021
05/26 11:17:09:  VxVM vxconfigd DEBUG  V-5-1-24847 setup_target_names: controller = 20 addr = w2612e000844db5a6,3e flags = 0xe notarg = 26021
05/26 11:17:09:  VxVM vxconfigd DEBUG  V-5-1-24847 setup_target_names: controller = 20 addr = w2612e000844db5a6,3d flags = 0xe notarg = 26021
05/26 11:17:09:  VxVM vxconfigd DEBUG  V-5-1-24847 setup_target_names: controller = 20 addr = w2612e000844db5a6,3c flags = 0xe notarg = 26021
05/26 11:17:09:  VxVM vxconfigd DEBUG  V-5-1-24847 setup_target_names: controller = 20 addr = w2612e000844db5a6,3b flags = 0xe notarg = 26021
<snippet>


The c20 controller paths are CLAIMED by the libvxhuawei.so ASL (Array-Support-Library)


# grep ddl_claim_single_disk vxconfigd_debug9.log | grep -w CLAIMED | grep "lib = " | grep "/dev/rdsk/c20t"
05/26 11:18:52:  VxVM vxconfigd DEBUG  V-5-1-14610 ddl_claim_single_disk: disk = /dev/rdsk/c20t26121C20DBEE054Ed32 lib = libvxhuawei.so DA stat = CLAIMED
05/26 11:18:55:  VxVM vxconfigd DEBUG  V-5-1-14610 ddl_claim_single_disk: disk = /dev/rdsk/c20t26021C20DBEE054Ed64 lib = libvxhuawei.so DA stat = CLAIMED
05/26 11:18:59:  VxVM vxconfigd DEBUG  V-5-1-14610 ddl_claim_single_disk: disk = /dev/rdsk/c20t2602E000844DB59Ed32 lib = libvxhuawei.so DA stat = CLAIMED
05/26 11:19:00:  VxVM vxconfigd DEBUG  V-5-1-14610 ddl_claim_single_disk: disk = /dev/rdsk/c20t2612E000844DB5A6d32 lib = libvxhuawei.so DA stat = CLAIMED
05/26 11:19:02:  VxVM vxconfigd DEBUG  V-5-1-14610 ddl_claim_single_disk: disk = /dev/rdsk/c20t2612E000844DB5A6d64 lib = libvxhuawei.so DA stat = CLAIMED
05/26 11:19:03:  VxVM vxconfigd DEBUG  V-5-1-14610 ddl_claim_single_disk: disk = /dev/rdsk/c20t26023C9D56E17026d32 lib = libvxhuawei.so DA stat = CLAIMED

 

# vxdmpadm listenclosure all
ENCLR_NAME        ENCLR_TYPE     ENCLR_SNO      STATUS       ARRAY_TYPE     LUN_COUNT    FIRMWARE
===================================================================================================
huawei-xsg10      HUAWEI-XSG1    2100e000844db59e     CONNECTED    ALUA       72         4301


In this example, the newly presented paths do not have an expected array_type setting of ALUA, so the newly discovered paths which are A/A are EXCLUDED:


# grep ddl_remove_other_atype vxconfigd_debug9.log
05/26 11:35:40:  VxVM vxconfigd DEBUG  V-5-1-28115 ddl_remove_other_atype: Enclosure with serial number 2100e000844db59e, array name HUAWEI-XSG1 and array type A/A is being removed from new device tree


The c20 controller paths are initially built and then excluded later in the discovery sequence:


# grep ddl_update_device_status vxconfigd_debug9.log-200526111536 | grep c20t | more

<snippet>
05/26 11:18:54:  VxVM vxconfigd DEBUG  V-5-1-0 ddl_update_device_status: Updating c20t26121C20DBEE054Ed32 (0x3d83500) -> 0 (libvxhuawei.so)
05/26 11:18:56:  VxVM vxconfigd DEBUG  V-5-1-0 ddl_update_device_status: Updating c20t26021C20DBEE054Ed64 (0x3d83c00) -> 0 (libvxhuawei.so)
05/26 11:19:02:  VxVM vxconfigd DEBUG  V-5-1-0 ddl_update_device_status: Updating c20t2612E000844DB5A6d32 (0x3d83700) -> 0 (libvxhuawei.so)
05/26 11:19:04:  VxVM vxconfigd DEBUG  V-5-1-0 ddl_update_device_status: Updating c20t26123C9D56E17026d30 (0x3d83910) -> 0 (libvxhuawei.so)
05/26 11:19:07:  VxVM vxconfigd DEBUG  V-5-1-0 ddl_update_device_status: Updating c20t26023C9D56E17026d32 (0x3d84100) -> 0 (libvxhuawei.so)
.
.
05/26 11:35:50:  VxVM vxconfigd DEBUG  V-5-1-0 ddl_update_device_status: Updating c20t2612E000844DB5A6d9 (0x3d837b8) -> 2 ()
05/26 11:35:50:  VxVM vxconfigd DEBUG  V-5-1-0 ddl_update_device_status: Updating c20t2612E000844DB5A6d8 (0x3d837c0) -> 2 ()
05/26 11:35:50:  VxVM vxconfigd DEBUG  V-5-1-0 ddl_update_device_status: Updating c20t2612E000844DB5A6d7 (0x3d837c8) -> 2 ()
05/26 11:35:50:  VxVM vxconfigd DEBUG  V-5-1-0 ddl_update_device_status: Updating c20t2612E000844DB5A6d64 (0x3d83600) -> 2 ()
<snippet>

The above messages indicate the c20 controller paths have been excluded by the "-> 2" value.



 

Solution


The HUAWEI-XSG1 enclosure has an array_type mode of ALUA. The array_type must be the same for all paths presented from a single enclosure.
 

# vxdmpadm listenclosure all
ENCLR_NAME        ENCLR_TYPE     ENCLR_SNO      STATUS       ARRAY_TYPE     LUN_COUNT    FIRMWARE
===================================================================================================
huawei-xsg10      HUAWEI-XSG1    2100e000844db59e     CONNECTED    ALUA       72         4301
 

If an array_type mismatch exists, the newly discovered paths/controllers will be marked as EXCLUDED and will not be seen by VxVM & DMP for that enclosure.


# grep ddl_remove_path_from_table vxconfigd_debug9.log | more
<snippet>
05/26 11:35:40:  VxVM vxconfigd DEBUG  V-5-1-25419 ddl_remove_path_from_table: devno of the path to be removed is : 3d83500
05/26 11:35:40:  VxVM vxconfigd DEBUG  V-5-1-25419 ddl_remove_path_from_table: devno of the path to be removed is : 3d83c00
05/26 11:35:40:  VxVM vxconfigd DEBUG  V-5-1-25419 ddl_remove_path_from_table: devno of the path to be removed is : 3d83700
05/26 11:35:40:  VxVM vxconfigd DEBUG  V-5-1-25419 ddl_remove_path_from_table: devno of the path to be removed is : 3d83910
<snippet>

The above DMP paths are removed from the DMP kernel.

The DMP devno (i.e. 3d83500) can be mapped back to the OS device name as follows (in this instance, c20t26121C20DBEE054Ed32)


# grep 3d83500 vxconfigd_debug9.log
05/26 11:17:19:  VxVM vxconfigd DEBUG  V-5-1-15138 setup_target_names: EFI device c20t26121C20DBEE054Ed32 devno 0x3d83500
05/26 11:18:16:  VxVM vxconfigd DEBUG  V-5-1-14097  c20t26121C20DBEE054Ed32 devno 0x3d83500 dev 246/13568 flags 0x802 type 3
05/26 11:18:53:  VxVM vxconfigd DEBUG  V-5-1-14145 copy_pathnode_info: Copying lun no. 6E000841004DB59E4E136DE100000067 having 0x3d83500 devno
05/26 11:18:54:  VxVM vxconfigd DEBUG  V-5-1-0 ddl_update_device_status: Updating c20t26121C20DBEE054Ed32 (0x3d83500) -> 0 (libvxhuawei.so)
05/26 11:35:40:  VxVM vxconfigd DEBUG  V-5-1-25419 ddl_remove_path_from_table: devno of the path to be removed is : 3d83500
05/26 11:35:51:  VxVM vxconfigd DEBUG  V-5-1-0 ddl_update_device_status: Updating c20t26121C20DBEE054Ed32 (0x3d83500) -> 2 ()


To address this issue, all paths from the single storage array must have the same array_type settings, in this instance, ALUA.

Once the backend settings have been set correctly for the impacted enclosure, rescan for the storage changes. This can be done on Solaris and Linux platforms using the DMP tool as follows:

# /usr/lib/vxvm/voladm.d/bin/dmpdr -o refresh

The enhanced dmpdr tool will only work in connection with Veritas DMP (Dynamic Multi-Pathing). Third-party multi-pathing drivers ( such as MPxIO and EMC PowerPath ) are not supported and will not work with interface.

 

 

Was this content helpful?