Veritas Access Installation Guide

How LLT supports RDMA for faster interconnections between applications

Low Latency Transport (LLT) maintains two channels (RDMA and non-RDMA) for each of the configured RDMA links. Both RDMA and non-RDMA channels can transfer data between the nodes. LLT provides separate Application Program Interfaces (APIs) to the clients (such as CFS and CVM) to use these channels. The RDMA channel is mainly used for data transfer by the client; while the non-RDMA channel is created over the UDP layer, and LLT uses it mainly for sending and receiving heartbeats. Group Membership Services/Atomic Broadcast (GAB) decides cluster membership for the cluster according to the health of the non-RDMA channel. The connections of the RDMA and non-RDMA channels are under separate management, while the connect and disconnect operations for the RDMA channel are triggered based on the status of the non-RDMA channel.

If the non-RDMA channel is up while the RDMA channel is down, the data is transferred over the non-RDMA channel with lower performance until the RDMA channel is fixed. The system logs display a message when the RDMA channel is up or down.

LLT uses the Open Fabrics Enterprise Distribution (OFED) layer and the drivers on the operating system to communicate with the hardware. LLT over RDMA allows applications running on one node to directly access the memory of an application running on another node over an RDMA-enabled network. While over a non-RDMA network, LLT clients have to create intermediate data copies to complete the read or write operation on the application. The RDMA network brings low latency, higher throughput, and minimized CPU host usage, and boosts application performance. LLT and GAB clients CFS and CVM can use LLT over RDMA.