Problem
vxiod processes consuming high CPU and causing the server to hang intermittently
Error Message
vxiod shows below stack.
#3 [] end_repeat_nmi at
[exception RIP: native_queued_spin_lock_slowpath+290]
#4 [] native_queued_spin_lock_slowpath at
#5 [] queued_spin_lock_slowpath at
#6 [] _raw_spin_lock_irqsave at
#7 [] volrp_sendsio_start at
Example:
PID: 4909 TASK: ffff907ac2920000 CPU: 6 COMMAND: "vxiod"
#0 [fffffe78f086ee60] crash_nmi_callback at ffffffff9ee734c1
#1 [fffffe78f086ee68] nmi_handle at ffffffff9ee36fbb
#2 [fffffe78f086eeb0] default_do_nmi at ffffffff9fb22560
#3 [fffffe78f086eed0] exc_nmi at ffffffff9fb22760
#4 [fffffe78f086eef0] end_repeat_nmi at ffffffff9fc0163c
[exception RIP: native_queued_spin_lock_slowpath+50]
RIP: ffffffff9fb37b12 RSP: ffffaa59c2063df0 RFLAGS: 00000002
RAX: 0000000000000001 RBX: ffff907f466e8000 RCX: 0000000000000000
RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff907cc1c84838
RBP: ffff907cc1c84838 R8: ffff907ca2f29c00 R9: 000000008020001a
R10: 0000000040000000 R11: ffff907fff82e600 R12: 0000000000000246
R13: ffff907cc1c84838 R14: 0000000000000000 R15: ffff907acab80000
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
--- <NMI exception stack> ---
#5 [ffffaa59c2063df0] native_queued_spin_lock_slowpath at ffffffff9fb37b12 <<<< spin lock here.
#6 [ffffaa59c2063e10] _raw_spin_lock_irqsave at ffffffff9fb372e0
#7 [ffffaa59c2063e20] vol_rv_send_ack_start at ffffffffc1822690 [vxio]
#8 [ffffaa59c2063ea8] voliod_iohandle at ffffffffc168e2cd [vxio]
#9 [ffffaa59c2063ee8] voliod_loop at ffffffffc168e5dd [vxio]
#10 [ffffaa59c2063f18] kthread at ffffffff9ef3eaad
#11 [ffffaa59c2063f50] ret_from_fork at ffffffff9ee08549
Cause
Due to spinlock contention, vxiod processes with a similar stack are consuming high CPU when 64+ VVR connections are configured. This leads to CPU starvation, causing the server to intermittently enter a hang state.
Solution
The Product Engineering team currently plans to address this issue through a patch or hotfix in the current software version. Please note that our company reserves the right to withdraw any fix from the targeted release if it fails quality assurance tests. Development plans are subject to change, and any actions you take based on this information, or your reliance on it, are at your own risk.
The code change has been modified to reduce the spinlock contention included in VRTSvxvm 8.0.2.2607 HF. This fix should be installed on top of either the refreshed or non-refreshed InfoScale 8.0.2u6 patch (ie InfoScale 8.0.2.3200 )
Public links to access this patch:
RHEL 8 - https://www.veritas.com/content/support/en_US/downloads/update.UPD951237
RHEL 9 - https://www.veritas.com/content/support/en_US/downloads/update.UPD213702
SLES 15 - https://www.veritas.com/content/support/en_US/downloads/update.UPD717762
Note:
- Arctera Infoscale Version 8.0.2U6 (build 8.0.2.3200) was refreshed on 13th September 2025 to incorporate critical VxFS updates, including VRTSvxfs-8.0.2.2800 and VRTSodm-8.0.2.2800. Please use the updated build for upgrade.
- The patch identified as VRTSvxvm 8.0.2.2607 has some additional bug fixes. Refer to its README for more information.