How to configure heartbeat communication for Veritas Cluster Server, including NIC Teaming

Description

This article contains configuration recommendations for Veritas Cluster Server (VCS) heartbeat communication.

VCS utilizes both public and private network connections for communication between clustered nodes and applications that are available to the public.

• Public network: Links are utilized for access to clustered applications.
• Private network: Links are used for heartbeat communication, as well as sharing status and updating information between clustered nodes. These private links are used for cluster communication only. No other application or resource should be configured to share these links. This may interfere with cluster communication and cause issues with cluster functionality.

The Low Latency Transfer (LLT) protocol uses heartbeat communication to transmit the cluster status and configuration information between nodes. This allows the Global Atomic Broadcast (GAB) component of the cluster to send and receive these updates among all cluster nodes. This communication is vital to the well-being of the cluster. If an active cluster node is unable to communicate with the other passive nodes, this may cause minor, to severe, issues with cluster behavior and performance. This may impede access to clustered applications.
 

Recommended Veritas Cluster Heartbeat Configurations

Use the following recommended configuration to set up VCS heartbeat communication. This ensures that there is no single point of failure, and that the private communication will function optimally.

• Do not assign IP addresses to private heartbeat network adapters. 
• Ensure that all adapters, and switch ports, are set to auto-negotiate, or are set to use matching speed settings. 
• All private heartbeats should be connected via a network hub or a switch. The best practice is to use only one hub, or switch, per heartbeat connection. This avoids a single point of failure.  
• Although cross-linked cables can be used for heartbeat purposes, we recommend that hubs, or switches, be used to ensure reliable connection.
• Ensure that the same driver revision is used across all NICs.
• Ensure that the same make and model of NICs are used throughout the network, to promote consistency.
• The public network connection should also have its own NIC and hub, or switch. We recommend using the same NIC make and model, and the same NIC driver be used for the public network, that are used for the private heartbeat connections. 
• If using three, or more, network adapters, set at least two of them as private heartbeat adapters in any manual configuration, or when using the Veritas Cluster Configuration Wizard. If required, set the third as a low priority (lowpri) heartbeat adapter, which uses the public network if both private heartbeats fail. This  maintains cluster communication over the public network, as a low priority, until the private heartbeat links can be restored.

Important Notice

• Network Routers are not supported for a private heartbeat network, nor are any other "Layer 3 aware" devices.
• Third party NIC teaming for Windows 2003, and earlier, is not supported for the private network heartbeat links. NIC teaming can only be implemented on the public network adapters.
  
  Note: This does not mean that NIC teaming cannot be configured successfully within a clustered environment for heartbeat communication. We do not test, document, support or assist in the configuration of NIC teaming for heartbeat communication. Should an issue arise from attempts to use NIC teaming for heartbeat communication on the private network, we recommend disabling NIC teaming and attempting to configure the private network links using a individual NICs.

• Native NIC team teaming for Windows Servers is supported. Note that only "Static Teaming" mode is currently supported, as per the "Cluster Configuration Wizard" section in the VCS Administrator Guide.

• LLT over UDP is supported with, or without, native OS NIC teaming because it is abstracted from the physical NIC configuration.