Problem
Mounting a vxfs file system complains that the filesystem is not clean and hence sets the fullfsck flag. Full FSCK completes successfully but does not clean the file system.
Error Message
# mount -F vxfs /dev/vx/rdsk/datadg/fs1 /mnt
UX:vxfs mount: ERROR: V-3-26881: Cannot be mounted until it has been cleaned by fsck. Please run "fsck -F vxfs -y /dev/vx/rdsk/datadg/fs1" before mounting
In syslog:
Jun 24 08:54:16 prod01 vxfs: [ID 702911 kern.warning] WARNING: msgcnt 2 mesg 021: V-2-21: vx_fs_init - /dev/vx/rdsk/datadg/fs1 file system validation failure
Cause
During mounting of filesystem, validation is done to check every fileset (FSET) entry in the Current Usage Table (CUT) matches with the valid fileset. In case of a cluster, a per node CUT files are maintained. During pass 4 of fsck, the dirty per node CUT files incore is merged and validated. There was a defect in fsck, where the merged CUT files were not being validated. Because of this defect, the corrupted CUT entries were not reconstructed. This defect is tracked via e2781552.
Background of CUT :
The Current Usage Table (CUT) is a file that contains usage-related information for each fileset. The information contained in the CUT changes frequently and is not replicated. The information in the CUT can, however, be reconstructed from the inode list if the CUT is damaged.
The CUT file contains one entry per fileset. The CUT entry for a given fileset contains information such as the number of blocks currently used by the fileset.
How to validate the issue?
Mount command failure:
# mount -F vxfs /dev/vx/dsk/datadg/fs1 /mnt
UX:vxfs mount: ERROR: V-3-26881: Cannot be mounted until it has been cleaned by
fsck. Please run "fsck -F vxfs -y /dev/vx/rdsk/datadg/fs1" before mounting
FSCK marks the file system clean but actually does not clean the f/s:
# fsck -F vxfs /dev/vx/rdsk/datadg/fs1
log replay in progress
log replay failed to clean file system
file system is not clean, full fsck required
full file system check required, exiting ...
# fsck -F vxfs -o full -y /dev/vx/rdsk/datadg/fs1
log replay in progress
pass0 - checking structural files
pass1 - checking inode sanity and blocks
pass2 - checking directory linkage
pass3 - checking reference counts
pass4 - checking resource maps
OK to clear log? (ynq)y
flush fileset headers? (ynq)y
set state to CLEAN? (ynq)y
#
# fsck -F vxfs /dev/vx/rdsk/datadg/fs1
file system is clean - log replay is not required
#
Superblock shows:
flags 800 mod 0 clean 69
# /usr/sbin/fsdb -F vxfs /dev/vx/dsk/datadg/fs1
> 8192B.p S
super-block at 00000008.0000
magic a501fcf5 version 7
ctime 1211014391 214820 (Sat May 17 18:53:11 2008 EST)
log_version 12 logstart 0 logend 0
bsize 1024 size 104538112 dsize 104538112 ninode 0 nau 0
defiextsize 0 oilbsize 0 immedlen 96 ndaddr 10
aufirst 0 emap 0 imap 0 iextop 0 istart 0
bstart 0 femap 0 fimap 0 fiextop 0 fistart 0 fbstart 0
nindir 2048 aulen 32768 auimlen 0 auemlen 8
auilen 0 aupad 0 aublocks 32768 maxtier 15
inopb 4 inopau 0 ndiripau 0 iaddrlen 8 bshift 10
inoshift 2 bmask fffffc00 boffmask 3ff checksum ed33ea3d
free 27727366 ifree 0
efree 261698 306012 313903 97492 84527 54084 34573 20940 11249 6704 3087 1088
537 283 38 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
flags 800 mod 0 clean 69 <== NOTE
time 1340496700 119989 (Sun Jun 24 10:11:40 2012 EST)
oltext[0] 32 oltext[1] 1282 oltsize 1
iauimlen 1 iausize 4 dinosize 256
checksum2 0 checksum3 0
fsetqu
>
> cut
cut entries start at 0x0000000b.0000
cu_fsindex 0 cu_curused 0 cu_lversion 0 cu_hversion 0
cu_dcurused 0 cu_visused 0 cu_update 0 0 cu_flags 0
>
> listfset
fset index fset name
1 ATTRIBUTE
999 UNNAMED
> olt
OLT at 0x00000020.0000
[…]
OLT cut entry:
olt_type 3 olt_size 16 olt_cutino[6 77]
[…]
OLT free entry:
olt_type 1 olt_fsize 776
>
> 1fset.6i
inode structure at 0x00000011.0200
type IFCUT mode 10000000777 nlink 1 uid 0 gid 0 size 1024
atime 1340449142 0 (Sat Jun 23 20:59:02 2012 EST)
mtime 1340449142 0 (Sat Jun 23 20:59:02 2012 EST)
ctime 1340449142 0 (Sat Jun 23 20:59:02 2012 EST)
aflags 0 orgtype 1 eopflags 0 eopdata 0
fixextsize/fsindex 0 rdev/reserve/dotdot/matchino 0
blocks 1 gen 0 version 0 0 iattrino 0
dotdot 0 inattrino 0
de: 11 0 0 0 0 0 0 0 0 0
des: 1 0 0 0 0 0 0 0 0 0
ie: 0 0
ies: 0
>
> 11b.p 2 C
cut entries start at 0x0000000b.0000
cu_fsindex 0 cu_curused 0 cu_lversion 0 cu_hversion 0 <==CUT entry for fset 1 seems to be corrupted
cu_dcurused 0 cu_visused 0 cu_update 0 0 cu_flags 0
cu_fsindex 0 cu_curused 0 cu_lversion 0 cu_hversion 0 <==CUT entry for fset 999 seems to be corrupted
cu_dcurused 0 cu_visused 0 cu_update 0 0 cu_flags 0
> 1fset.77i
inode structure at 0x0000001b.0100
type IFCUT mode 10000000777 nlink 1 uid 0 gid 0 size 1024
atime 1211015426 792839 (Sat May 17 19:10:26 2008 EST)
mtime 1211015426 793778 (Sat May 17 19:10:26 2008 EST)
ctime 1211015426 793778 (Sat May 17 19:10:26 2008 EST)
aflags 0 orgtype 1 eopflags 0 eopdata 0
fixextsize/fsindex 0 rdev/reserve/dotdot/matchino 0
blocks 1 gen 1561544662 version 0 4 iattrino 0
dotdot 0 inattrino 0
de: 131080 0 0 0 0 0 0 0 0 0
des: 1 0 0 0 0 0 0 0 0 0
ie: 0 0
ies: 0
>
> 131080b.p 2 C
cut entries start at 0x00020008.0000
cu_fsindex 1 cu_curused 0 cu_lversion 0 cu_hversion 0
cu_dcurused 0 cu_visused 0 cu_update 0 0 cu_flags 0
cu_fsindex 999 cu_curused 0 cu_lversion 0 cu_hversion 0
cu_dcurused 0 cu_visused 0 cu_update 0 0 cu_flags 0
>
Once the fix is installed:
# ./fsck -F vxfs -o full -y /dev/vx/dsk/datadg/fs1
pass0 - checking structural files
pass1 - checking inode sanity and blocks
pass2 - checking directory linkage
pass3 - checking reference counts
pass4 - checking resource maps
no CUT entry for fileset 1, fix? (ynq)y
no CUT entry for fileset 999, fix? (ynq)y
OK to clear log? (ynq)y
flush fileset headers? (ynq)y
set state to CLEAN? (ynq)y
#
Mount now works ok:
# mount -F vxfs /dev/vx/dsk/datadg/fs1 /mnt
#
Checking superblock and CUT entries:
# fsdb -F vxfs /dev/vx/dsk/datadg/fs1
> cut
cut entries start at 0x0000000b.0000
cu_fsindex 1 cu_curused 0 cu_lversion 0 cu_hversion 0
cu_dcurused 0 cu_visused 0 cu_update 0 0 cu_flags 0
> 1fset
fset header structure at 0x00000009.0000
fsh_fsindex 1 fsh_fsetname "STRUCTURAL"
fsh_version 6 fsh_checksum 0xd8c1a596
fsh_time 1213935850 175030 (Thu Jun 19 21:24:10 2008 MDT)
fsh_ninode 128 fsh_nau 1 fsh_old_ilesize 0 fsh_eopdata 0
fsh_fsextop 0x0 fsh_dflags 0xa fsh_quota 0 fsh_maxinode 4294967295
fsh_ilistino[5 37] fsh_iauino 4 fsh_lctino 0 fsh_fclino 0
fsh_uquotino 0 fsh_gquotino 0
fsh_attr_ninode 0 fsh_attr_nau 0 fsh_attr_eopdata 0
fsh_attr_ilistino[0 0] fsh_attr_iauino 0 fsh_attr_lctino 0
fsh_features 0x1
fsh_previx 0 fsh_nextix 0
fsh_ctime 1211014391 214820 (Sat May 17 01:53:11 2008 MDT)
fsh_mtime 1211014391 214820 (Sat May 17 01:53:11 2008 MDT)
> 6i
inode structure at 0x00000011.0200
type IFCUT mode 10000000777 nlink 1 uid 0 gid 0 size 1024
atime 1340449142 0 (Sat Jun 23 03:59:02 2012 MDT)
mtime 1340449142 0 (Sat Jun 23 03:59:02 2012 MDT)
ctime 1340449142 0 (Sat Jun 23 03:59:02 2012 MDT)
aflags 0 orgtype 1 eopflags 0 eopdata 0
fixextsize/fsindex 0 rdev/reserve/dotdot/matchino 0
blocks 1 gen 0 version 0 0 iattrino 0
dotdot 0 inattrino 0
de: 11 0 0 0 0 0 0 0 0 0
des: 1 0 0 0 0 0 0 0 0 0
ie: 0 0
ies: 0
> 11b.p 2 C
cut entries start at 0x0000000b.0000
cu_fsindex 1 cu_curused 0 cu_lversion 0 cu_hversion 0
cu_dcurused 0 cu_visused 0 cu_update 0 0 cu_flags 0
cu_fsindex 999 cu_curused 0 cu_lversion 0 cu_hversion 0
cu_dcurused 0 cu_visused 0 cu_update 0 0 cu_flags 0
> 77i
inode structure at 0x0000001b.0100
type IFCUT mode 10000000777 nlink 1 uid 0 gid 0 size 1024
atime 1211015426 792839 (Sat May 17 02:10:26 2008 MDT)
mtime 1211015426 793778 (Sat May 17 02:10:26 2008 MDT)
ctime 1211015426 793778 (Sat May 17 02:10:26 2008 MDT)
aflags 0 orgtype 1 eopflags 0 eopdata 0
fixextsize/fsindex 0 rdev/reserve/dotdot/matchino 0
blocks 1 gen 1561544662 version 0 4 iattrino 0
dotdot 0 inattrino 0
de: 131080 0 0 0 0 0 0 0 0 0
des: 1 0 0 0 0 0 0 0 0 0
ie: 0 0
ies: 0
> 131080b.p 2 C
cut entries start at 0x00020008.0000
cu_fsindex 1 cu_curused 0 cu_lversion 0 cu_hversion 0
cu_dcurused 0 cu_visused 0 cu_update 0 0 cu_flags 0
cu_fsindex 999 cu_curused 0 cu_lversion 0 cu_hversion 0
cu_dcurused 0 cu_visused 0 cu_update 0 0 cu_flags 0
The CUT entries in both inodes 6 and 77 are now consistent.
Solution
Veritas Engineering has made the necessary code changes to validate the merged CUT file. It now checks whether for every fileset, there is a valid CUT entry. For a fileset, if there is no CUT entry, a CUT entry is created for it.
The fix is currently available in the following releases:
SFHA 5.1 & 6.0.5
Please contact Veritas Enterprise Support if a fix is needed for other platforms/releases.
Applies To
Applicable to all platforms with VxFS 5.1SP1 and 6.0.5
References
Was this content helpful?
Translated Content
Please note that this document is a translation from English, and may have been machine-translated. It is possible that updates have been made to the original version after this document was translated and published. Veritas does not guarantee the accuracy regarding the completeness of the translation. You may also refer to the English Version of this knowledge base article for up-to-date information.