Intermittent failures and possible system crashes are known to occur when using Storage Foundation for UNIX on certain AIX platforms

Problem

Intermittent failures and possible system crashes are known to occur when using Storage Foundation for UNIX, (SF/HA) Veritas Cluster Server, (VCS) Storage Foundation for Oracle RAC, (SFRAC) and Storage Foundation Cluster File Server (SFCFS) with certain AIX platforms.

Solution



Problem: System Crashes/Hangs in one of the drivers of the cluster solutions

System crash/hang will occur with any version of clustering products based on 4.0 or 5.0 releases. The Service Pack releases of AIX with the patch levels specified in the following table are the levels of AIX which result in the crash.

 

O/S LevelStorage FoundationVeritas Cluster ServerStorage Foundation High AvailabilityStorage Foundation Cluster File ServerStorage Foundation Oracle RAC
AIX 5.2 TL10 SP5NOYESYESYESYES
AIX 5.3 TL6 SP3NOYESYESYESYES
AIX 5.3 TL7 SP2NOYESYESYESYES
AIX 6.1 Base SP3NOYESYESYESYES



If your system is upgraded to or installed at the specified levels or above, you are impacted. You must follow the instructions provided below to recover. The specific IBM AIX APAR triggering this LLT panic has the abstract: Bring DLPI driver "to SPEC" applies to AIX - enhance DLPI driver to comply with specifications or normal streams processing. This APAR is installed as:

AIX OS LevelAIX APAR
5200-10IZ19838
5300-06IZ05430
5300-07IZ11726
6100-00IZ13304




Problem and fix description

Changes to the DLPI driver with the latest TLs on AIX 5.3 and 6.1 result in upcalls to LLT in the interrupt context instead of the process context. Some LLT locks are not interrupt safe which caused panics. Some upcalls panic in GAB where it does an xmalloc in interrupt context.

All LLT locks on AIX are now interrupt safe. Out-of order lock releases are handled correctly with the new interrupt safe locks. Link notifications are provided for the client in the thread or process context instead of the interrupt context. This will ensure that no changes are required for LLT's clients like GAB/LMX thus abstracting all the changes to just one driver.


Impacted releases and corresponding LLT patches

ReleasePatch
4.0MP34.0MP3+e1274390_llt_b
4.0MP44.0MP4+e1274390_llt
5.05.0+e1274390_llt
5.0MP15.0MP1_5.0MP1EXT+e1274390_llt
5.0MP1-AIX6.15.0MP1_5.0MP1EXT+e1274390_llt



The corresponding LLT patches are found at Patch Central below and under the Related Documents section.

4.0 MP3    https://sort.symantec.com/patch/detail/2106

4.0 MP4    https://sort.symantec.com/patch/detail/2107

5.0            https://sort.symantec.com/patch/detail/2104

5.0 MP1 and 5.0 MP1 Update 1  https://sort.symantec.com/patch/detail/2103


Recommended procedure for an existing setup with panics

- Install and configure the appropriate patch for your MP


Recommended procedure for an existing setup without panics

- Install and configure the patch for the appropriate release
  (follow directions in the README)
- Upgrade to the latest AIX release when ready.


Recommended procedure for a fresh install

- Use installonly to install the base release
- Upgrade to the latest MP
- Install the required llt patch, then reboot the system.
- Use the configure option to configure the cluster-server.


NOTE#  The removal of IBM temporary patch. (Workaround)

Once the LLT patch is installed as above, the workaround patch can be uninstalled, if it was installed.

To uninstall the workaround patch:

For AIX 6.1:
   # emgr -rL 61TL00dlpi
For AIX 5.3 TL6:
   # emgr -rL 53TL06dlpi
For AIX 5.3 TL7:
   # emgr -rL 53TL07dlpi


 

 

 

Terms of use for this information are found in Legal Notices.

Search

Survey

Did this article answer your question or resolve your issue?

No
Yes

Did this article save you the trouble of contacting technical support?

No
Yes

How can we make this article more helpful?

Email Address (Optional)