blade servers form SFRAC cluster, LLT NIC working mode Auto-Negotiation often cause "network partition" and lead node panic

Problem

blade servers form SFRAC cluster, LLT NIC working mode Auto-Negotiation often cause "network partition" and lead node panic

Error Message

KERNEL: usr/lib/debug/lib/modules/2.6.18-164.el5/vmlinux  
   DUMPFILE: /var/crash/2012-04-24-18:48/vmcore  
       CPUS: 16  
       DATE: Tue Apr 24 18:44:47 2012  
     UPTIME: 00:24:12  
LOAD AVERAGE: 47.97, 17.43, 7.23  
      TASKS: 769  
   NODENAME: SBCJems2  
    RELEASE: 2.6.18-164.el5  
    VERSION: #1 SMP Fri Dec 3 08:56:42 CST 2010  
    MACHINE: x86_64  (2133 Mhz)  
     MEMORY: 23.6 GB  
      PANIC: "Kernel panic - not syncing: GAB: Port d halting system due to network failure at [14:2027]"  
        PID: 7691  
    COMMAND: "lltdlv"  
       TASK: ffff810314e0e860  [THREAD_INFO: ffff810314d6a000]  
        CPU: 11  
      STATE: TASK_RUNNING (PANIC)

     KERNEL: usr/lib/debug/lib/modules/2.6.18-164.el5/vmlinux  
   DUMPFILE: /var/crash/2012-04-23-07:20/vmcore  
       CPUS: 16  
       DATE: Mon Apr 23 07:15:39 2012  
     UPTIME: 3 days, 18:15:58  
LOAD AVERAGE: 105.29, 32.28, 12.47  
      TASKS: 1808  
   NODENAME: SBCJems2  
    RELEASE: 2.6.18-164.el5  
    VERSION: #1 SMP Fri Dec 3 08:56:42 CST 2010  
    MACHINE: x86_64  (2133 Mhz)  
     MEMORY: 23.6 GB  
      PANIC: "Kernel panic - not syncing: GAB: Port f halting system due to network failure at [14:2027]"  
        PID: 7889  
    COMMAND: "lltdlv"  
       TASK: ffff8104f819e040  [THREAD_INFO: ffff8104f61a0000]  
        CPU: 9  
      STATE: TASK_RUNNING (PANIC)

 

Cause

on blade server , all NIC connect through blade enclosure mainboard ,default working mode is Auto-Negotiation .

it should cause LLT NIC communication disconnect <--> connect  frequently

after kernel port rejoin to gab,  generation number conflict ,so I/O Fence panic the node

Solution

config all LLT NIC working mode to permanent mode , such as 100M Full Duplex , not using Auto-Negotiation 


Applies To

RHEL 5

SFRAC 5.1SP1

Terms of use for this information are found in Legal Notices.

Search

Survey

Did this article answer your question or resolve your issue?

No
Yes

Did this article save you the trouble of contacting technical support?

No
Yes

How can we make this article more helpful?

Email Address (Optional)