Procedure to be adopted for disk removal from diskgroup specific to thin reclamation luns

Problem

Error Message

Solution

Thin Reclamation lets an administrator trigger onlinereclamation of unused space in thin-reclaim enabled arrays. You can only performsuch operation on luns exhibiting thin_rclm attribute/flag. Volume Manager(VxVM) automatically discovers devices on arrays that support thin reclamation.To list devices that are known to be thin or thin_rclm on a host, use the vxdisk-o thin list command.

For example:

#vxdisk -o thin list

DEVICE     SIZE(mb)     PHYS_ALLOC(mb) AU(blk)MAXRCLM(mb)  GROUP   TYPE
tagmastore-usp0_384112288        126          860161344         asebsdb1dg thinrclm
tagmastore-usp0_384212288        126          860161344         -      thinrclm
tagmastore-usp0_384310240        126          860161344         asebsdb1dg thinrclm
tagmastore-usp0_384492160        168          860161344         asebsdb1dg thinrclm
tagmastore-usp0_384561440        210          860161344         asebsdb1dg thinrclm
tagmastore-usp0_384610240        126          860161344         asebsdb1dg thinrclm
tagmastore-usp0_38475120         126          860161344         asebsdb1dg thinrclm
tagmastore-usp0_38485120         4452         860161344         asebsdb1dg thinrclm
tagmastore-usp0_384925600       18354       860161344         asebsdb1dg thinrclm


When a volume isdeleted from a disk group, subdisks are not removed immediately but have aspecial flag set that marks them for deletion/reclamation at a specific time.The default scheduled cycle for thin_reclaim subdisks marked for reclamation isset to 22:10 every day. The "vxdefault list" outputs the default/current settings.

A couple of helpful commands that lists the subdisks with thereclaim flag set are :

1. vxprint -z which will list keyword RECLAIM  under the STATE column
2. vxprint with a combination ofoptions as shown below
$ vxprint -g tdg-lz
...
Subdisk:  xiv0_030a-01
info:     disk=xiv0_030aoffset=2048000 len=2097152
assoc:    vol=(dissoc)plex=(dissoc)
flags:    enabledreclaim_pnd  <<<<<<<<<<


Theadvantage of setting a convenient "reclaim_on_delete_start_time" is to allow forthe recovery of volumes if required (recovery is possible from the time ofdeletion and upto/just prior to 'reclaim_on_delete_start_time)


Tolist the current default values using the vxdefault command :

#vxdefault list
KEYWORD                        CURRENT-VALUE  DEFAULT-VALUE
usefssmartmove                thinonly        thinonly
fssmartmovethreshold          100            100
sharedminorstart              33000          33000
reclaim_on_delete_wait_period  1              1
reclaim_on_delete_start_time  22:10          22:10
usesmartmovewithvvr            on              on

To modify the reclamation time from 22:10 to 4:00

#vxdefault set  reclaim_on_delete_start_time 4:00

#vxdefault list
KEYWORD                        CURRENT-VALUE  DEFAULT-VALUE
usefssmartmove                thinonly        thinonly
fssmartmovethreshold          100            100
sharedminorstart              33000          33000
reclaim_on_delete_wait_period  1              1
reclaim_on_delete_start_time  4:00            22:10
usesmartmovewithvvr            on              on


Attempting to remove a disk with the "thin_rclm" flag set will result in the followingerror - if the command is executed anytime outside of the"reclaim_on_delete_start_time"  when the subdisks will have thereclaim_pnd(pending) flags set.

To workaround the issue - one can issue the "vxdisk reclaim <daname>" to force a reclamation immediately prior toexecuting the vxdg -g rmdisk command - as illustrated below .


!! The following vxprint output illustrates the disk being removed does not appear to have any subdisks used in active volumes

#vxprint -htg asebsdb1dg  

dg asebsdb1dg  default      default  14000    1268411694.67.asebsdb1

dmasebsdb1dg01 tagmastore-usp0_3841 auto 65536 24995600 -
dm asebsdb1dg02tagmastore-usp0_3842 auto 65536 24995600
dm asebsdb1dg03 tagmastore-usp0_3843auto 65536 20802640 -
dm asebsdb1dg04 tagmastore-usp0_3844 auto 65536188569424 -
dm asebsdb1dg05 tagmastore-usp0_3845 auto 65536 125626768 -
dmasebsdb1dg06 tagmastore-usp0_3846 auto 65536 20802640 -
dm asebsdb1dg07tagmastore-usp0_3847 auto 65536 10411776 -
dm asebsdb1dg08tagmastore-usp0_3848 auto 65536 10411776 -
dm asebsdb1dg09tagmastore-usp0_3849 auto 65536 52225776-


v  asebsdb1-archive1-vol -   ENABLED  ACTIVE  20801536 SELECT    -        fsgen
plasebsdb1-archive1-vol-01 asebsdb1-archive1-vol ENABLED ACTIVE 20801536 CONCAT -RW
sd asebsdb1dg03-01 asebsdb1-archive1-vol-01 asebsdb1dg03 0 20801536 0tagmastore-usp0_3843 ENA

v  asebsdb1-backup1-vol-    ENABLED  ACTIVE   188567552 SELECT  -        fsgen
pl asebsdb1-backup1-vol-01asebsdb1-backup1-vol ENABLED ACTIVE 188567552 CONCAT - RW
sd asebsdb1dg04-01asebsdb1-backup1-vol-01 asebsdb1dg04 0 188567552 0 tagmastore-usp0_3844ENA

v  asebsdb1-oracle-vol -    ENABLED  ACTIVE   24993792SELECT    -        fsgen
plasebsdb1-oracle-vol-01 asebsdb1-oracle-vol ENABLED ACTIVE 24993792 CONCAT -RW
sd asebsdb1dg01-01 asebsdb1-oracle-vol-01 asebsdb1dg01 0 24993792 0tagmastore-usp0_3841 ENA

v  asebsdb1-oradata1-vol -  ENABLED  ACTIVE   125626368 SELECT  -        fsgen
pl asebsdb1-oradata1-vol-01asebsdb1-oradata1-vol ENABLED ACTIVE 125626368 CONCAT - RW
sd asebsdb1dg05-01asebsdb1-oradata1-vol-01 asebsdb1dg05 0 125626368 0 tagmastore-usp0_3845ENA

v  asebsdb1-oradata2-vol -   ENABLED  ACTIVE  20801536 SELECT    -        fsgen
plasebsdb1-oradata2-vol-01 asebsdb1-oradata2-vol ENABLED ACTIVE 20801536 CONCAT -RW
sd asebsdb1dg06-01 asebsdb1-oradata2-vol-01 asebsdb1dg06 0 20801536 0tagmastore-usp0_3846 ENA

v  asebsdb1-oradata3-vol -  ENABLED  ACTIVE   10409984SELECT    -        fsgen
plasebsdb1-oradata3-vol-01 asebsdb1-oradata3-vol ENABLED ACTIVE 10409984 CONCAT -RW
sd asebsdb1dg07-01 asebsdb1-oradata3-vol-01 asebsdb1dg07 0 10409984 0tagmastore-usp0_3847 ENA


!! Despite the disk not having a subdiskassociated with any volume - the command fails as follows:

#vxdg  -g asebsdb1dg rmdisk asebsdb1dg02
VxVM vxdg ERRORV-5-1-553 Disk asebsdb1dg02 is used by one or more subdisks


#vxdisk list
DEVICE      TYPE            DISK        GROUP        STATUS
tagmastore-usp0_3840auto:LVM        -            -            LVM
tagmastore-usp0_3841auto:cdsdisk    asebsdb1dg01  asebsdb1dg   onlinethinrclm
tagmastore-usp0_3842auto:cdsdisk    asebsdb1dg02  asebsdb1dg   onlinethinrclm
:
:

Reclaim the device.
# vxdisk -g asebsdb1dg reclaim tagmastore-usp0_3842
Disk tagmastore-usp0_3842 : Skipped. No VxFS file systemfound.

#vxdisk list
DEVICE      TYPE            DISK        GROUP        STATUS
tagmastore-usp0_3840auto:LVM        -            -            LVM
tagmastore-usp0_3841auto:cdsdisk    asebsdb1dg01  asebsdb1dg   onlinethinrclm
tagmastore-usp0_3842auto:cdsdisk    asebsdb1dg02  asebsdb1dg   onlinethinrclm
:
:

!! Notice the disk now has been successfully removedfrom diskgroup "asebsdb1dg"

#vxdg  -g asebsdb1dg rmdisk asebsdb1dg02
#vxdisk list
DEVICE      TYPE            DISK        GROUP        STATUS
tagmastore-usp0_3840auto:LVM        -            -            LVM
tagmastore-usp0_3841auto:cdsdisk    asebsdb1dg01  asebsdb1dg   onlinethinrclm
tagmastore-usp0_3842auto:cdsdisk    -            -            onlinethinrclm
:
: