About tuning Veritas File System on AIX
The following list describes key AIX-specific characteristics of Veritas File System (VxFS) that are relevant in tuning for improved buffered I/O performance:
The first mount of a VxFS file system on the AIX platform allocates Virtual Memory Management (VMM) structures, specifically Page Device Table (PDT) structures and VMM buffers.
The last unmount of a VxFS file system de-allocates these VMM structures.
Between 1 and 64 VxFS PDT structures and many thousands of VxFS VMM buffers are created and the default values are autotuned based on the system configuration.
When an inode is brought in-core, a PDT is associated with the inode. The PDT is selected on a round-robin basis.
All buffered I/Os to and from a given file are performed using the VMM buffers attached to the PDT that is associated to the file's in-core inode.
The VMM buffers are only used to perform buffered I/O: either the paging-in or paging-out of client pages.
VxFS 5.1 and later can function in D_REFUND mode (drefund_enable=1). However, D_REFUND is supported (drefund_supported=1) only on AIX 6.1 TL2 and later.
The vxtunefs command can be used to determine the current values for drefund_supported and drefund_enable.
In the absence of D_REFUND mode, the number of PDTs are automatically tuned to 4 times the number of CPUs, up to 64. The total number of VMM buffers are autotuned similarly based on the amount of physical memory of the system. The VMM buffers are then evenly distributed across the PDTs. Hence, if there are either fewer PDTs or a greater number of VMM buffers, then the number of VMM buffers per PDT is larger. The following vxtunefs commands can be used to view the number PDTs and VMM buffers on the system:
# /opt/VRTS/bin/vxtunefs -D print | grep num_pdt
# /opt/VRTS/bin/vxtunefs -b
In D_REFUND mode, the number of VMM buffers on a PDT can be dynamically grown and shrunk based on the demand, unlike the static distribution in absence of D_REFUND mode.
D_REFUND mode faces the limitation of having only 50,000 VMM buffer to allocate dynamically across the PDTs. This can be a limitation in some cases where D_REFUND should be turned off deliberately followed by the appropriate tuning for number of PDTs and VMM buffers.
Number of PDTs and VMM buffers should only be tuned in absence of D_REFUND.
In the absence of D_REFUND, VxFS performs accounting by allowing the reservation and un-reservation of VMM buffers that are associated with a PDT, before performing an I/O. No such accounting is performed in D_REFUND mode.
Such accounting can sometimes lead to frequent memory allocation, which causes high CPU usage by vx_sched. In such scenarios, you can disable the accounting by setting the vmmbufs_resv_disable tunable using the vxtunefs command.
See Advice for tuning Veritas File System on AIX