Server panicked (Panic String : BAD TRAP: type=31 rp=2a101f4d480 addr=8 mmu_fsr=0 occurred in module "vxdmp" due to a NULL pointer dereference)


 Server was panicked with the following panic string :

BAD TRAP: type=31 rp=2a101f4d480 addr=8 mmu_fsr=0 occurred in module "vxdmp" due to a NULL pointer dereference


The panic occurs as the inconsistence of dmpdb/devlist between vxesd and vold/vxdmp. When panic occurs, because of SAN outage/restore, devices were lost then detected in vxesd but they were not in vold/vxdmp. Thus, if we issue device map updates from vxesd, the dmpnode may do not exists in vxdmp, panic occurs.






 Analysis of Crash Dump : SolarisCAT(415-098-321_vmcore.2/10U)> panic

 panic on CPU 9

panic string:   BAD TRAP: type=31 rp=2a101f4d480 addr=8 mmu_fsr=0 occurred in module "vxdmp" due to a NULL pointer dereference

==== panic user (LWP_SYS) thread: 0x3000baca740  PID: 561  on CPU: 9  affinity CPU: 9 ====

cmd: /sbin/vxesd

t_procp: 0x6007da08050

  p_as: 0x60069fb4910  size: 945283072  RSS: 135831552

  hat: 0x30010402700

    cnum: CPU0:2463/371 CPU2:2111/140 CPU4:2066/131 CPU6:2099/133 CPU8:2713/251 CPU10:2260/109 CPU12:2209/270 CPU14:2231/138

    cpusran: 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15

  zone: global

t_stk: 0x2a101f4dae0  sp: 0x18c0791  t_stkbase: 0x2a101f48000

t_pri: 59(TS)  t_tid: 1  pctcpu: 0.010421

t_lwp: 0x600637880a0  machpcb: 0x2a101f4dae0

  mstate: LMS_SYSTEM  ms_prev: LMS_KFAULT

  ms_state_start: 0.0000741 seconds earlier

  ms_start: 190 days 1 hours 57 minutes 5.3898810 seconds earlier

psrset: 0  last CPU: 9

idle: 0 ticks (0 seconds)

start: Sat Jan 29 00:08:16 2011

age: 16422865 seconds (190 days 1 hours 54 minutes 25 seconds)

syscall: #54 ioctl(, 0xffbffb20) (sysent: genunix:ioctl+0x0)

tstate: TS_ONPROC - thread is being run on a processor

tflg:   T_PANIC - thread initiated a system panic

        T_DFLTSTK - stack is default size

tpflg:  TP_TWAIT - wait to be freed by lwp_wait

        TP_MSACCT - collect micro-state accounting information

tsched: TS_LOAD - thread is in memory

        TS_DONT_SWAP - thread/LWP should not be swapped

pflag:  SMSACCT - process is keeping micro-state accounting

        SMSFORK - child inherits micro-state accounting


pc:      unix:panicsys+0x48:   call     unix:setjmp


unix:panicsys+0x48(0x10a4bb8, 0x2a101f4d228, 0x18c1160, 0x1, , , 0x9900001607, , , , , , , , 0x10a4bb8, 0x2a101f4d228)

unix:vpanic_common+0x78(0x10a4bb8, 0x2a101f4d228, 0x36921, 0x2a101f4cd21, 0x2a101f4d520, 0x4480001602)

unix:panic+0x1c(0x10a4bb8, 0x31, 0x2a101f4d480, 0x8, 0x0, 0x30009fb6240, 0x183a368)

unix:die+0x78(0x31, 0x2a101f4d480, 0x8, 0x0)

unix:trap+0x9e0(0x2a101f4d480, 0x8)


-- trap data  type: 0x31 (data access MMU miss)  rp: 0x2a101f4d480  --

  addr: 0x8

pc:  0x7be5c688 vxdmp:dmp_decode_modmap_dmpnode+0x1c4:   ldx    [%l4 + 0x8], %o2

npc: 0x7be5c68c vxdmp:dmp_decode_modmap_dmpnode+0x1c8:   subcc  %o2, 0x0, %g0   ( cmp   %o2, 0x0             )

  global:                       %g1         0x7be5c678

        %g2              0x200  %g3           0x41798f

        %g4          0x19bd000  %g5             0x19bd

        %g6                  0  %g7      0x3000baca740

  out:  %o0             0x19bf  %o1             0x1800

        %o2            0x10080  %o3              0x200

        %o4          0x10a1000  %o5             0x10a1

        %sp      0x2a101f4cd21  %o7         0x7be5c680

  loc:  %l0      0x11800000008  %l1                  0

        %l2           0x100003  %l3                0x9

        %l4                  0  %l5         0x375f6070

        %l6                  0  %l7                0x1

  in:   %i0      0x2a101f4d698  %i1           0x100003

        %i2         0x375f6064  %i3          0x19b8cb8

        %i4                  0  %i5                  0

        %fp      0x2a101f4cde1  %i7         0x7be576d0

<trap>vxdmp:dmp_decode_modmap_dmpnode+0x1c4(0x2a101f4d698, 0x100003, 0x0, 0x19b8cb8)

vxdmp:dmp_decipher_instructions+0x364(0x2a101f4d758, 0x4f, 0x100003, 0x0, 0x19b8cb8)

vxdmp:dmp_process_instruction_buffer+0x150(0x1180003ffff, 0x78ae8, 0x100003, 0x0, 0x19b8cb8)

vxdmp:dmp_reconfigure_db+0x48(, 0x78ae8, 0x100003)

vxdmp:gendmpioctl(0x1180003ffff, , 0x78ae8, 0x100003, 0x6007d831ee0, 0x2a101f4dadc) - frame recycled

vxdmp:dmpioctl+0x20(, 0x444d5040, 0x78ae8, 0x100003, 0x6007d831ee0)

specfs:spec_ioctl(0x60069b53500, 0x444d5040, 0x78ae8, 0x100003, 0x6007d831ee0) - frame recycled

genunix:fop_ioctl+0x20(0x60069b53500, 0x444d5040, 0x78ae8, 0x100003, , 0x2a101f4dadc)



-- switch to user thread's user stack --


Found etrack 2141176: Panic: BAD TRAP: type=31 in dmp_decode_modmap_dmpnode()

HF4 for 5.0MP3RP4 contains the fix for this issue. Please contact Symantec Technical support for this Hot Fix.

Workaround for this issue :

/sbin/vxesd -k -x dmp_monitor_fabric=off 



Applies To

 OS Version : Solaris 10

SF Version : 5.0MP3RP4

Terms of use for this information are found in Legal Notices.



Did this article answer your question or resolve your issue?


Did this article save you the trouble of contacting technical support?


How can we make this article more helpful?

Email Address (Optional)