Disable Transparent Huge Page on NBU media/storage server to ensure better NetBackup workloads performance
Problem
With THP enabled, NetBackup appliance would perform poorly or become irresponsive when Operating System is under memory pressure, for example:- System is constantly running with heavy workload
- Spoold takes a large percent of memory, which is usually related to large MSDP file systems and high usage rate of the file systems
Cause
The Transparent Huge Pages feature is enabled by default in RHEL/CentOS 6 or 7. The kernel will always attempt to satisfy a high-order memory allocation using hugepages. If no hugepages are available, the kernel will try to defrag memory to get hugepages. This defrag effort is time-consuming when system is under memory pressure and will cause high latency to user-land processes.Solution
NOTE: The following recommended Linux tunable to disable THP is for supported Linux systems dedicated for NetBackup services. If there are other major services or applications operating on or sharing system resources, please confirm that disabling THP has no negative impact on other 3rd party application or services.
NOTE 2: NetBackup Appliance 3.1 and later have THP disabled by default.
Check THP status with the following command. THP is enabled if the output is always or madvise
[always] madvise never
# cat /sys/kernel/mm/transparent_hugepage/enabled
always [madvise] never
Disable THP
- Disable THP without rebooting
# cat /sys/kernel/mm/transparent_hugepage/enabled
always madvise [never]
This change will not affect existing hugepages but future behaviors
- Disable THP at boot time by adding transparent_hugepage=never to the end of the kernel line and rebooting
For RHEL 6 or Appliance 3.0 or Less
kernel /vmlinuz-2.6.32-504.30.3.el6.x86_64 ro root=/dev/mapper/system-root rd_NO_LUKS KEYBOARDTYPE=pc KEYTABLE=us LANG=en_US.UTF-8 rd_LVM_LV=system/root nodmraid rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=256M rd_LVM_LV=system/swap nomodeset rdblacklist=qla2xxx rd_NO_DM rdblacklist=qla2xxx loglevel=1 transparent_hugepage=never
For RHEL 7 or Appliance 3.1 or higher
1. Add the "transparent_hugepage=never" kernel parameter option to the grub2 configuration file.
- Edit the /etc/default/grub file as shown below
GRUB_CMDLINE_LINUX="nomodeset rdblacklist=qla2xxx crashkernel=auto rd.lvm.lv=system/root rd.lvm.lv=system/swap nodmraid rhgb quiet audit=1 transparent_hugepage=never"
2. Rebuild the /boot/grub2/grub.cfg file by running the grub2-mkconfig -o command as follows:
- Please ensure to take a backup of the existing
/boot/grub2/grub.cfg
before rebuilding. (NetBackup Appliances are currently BIOS-based)- On BIOS-based machines:
~]# grub2-mkconfig -o /boot/grub2/grub.cfg
- On UEFI-based machines:
~]# grub2-mkconfig -o /boot/efi/EFI/redhat/grub.cfg
- On BIOS-based machines: