A backup attempting to use fiber transport may fail with a status 83: could not open FT Server pipe: pipe open failed.

Problem

A backup attempting to use fiber transport may fail with a status 83: could not open FT Server pipe: pipe open failed. A status 83 may have multiple underlying causes and requires further investigation. In this case, the status 83 was caused by a data buffer size too large for the DMAbuffer to be allocated.

Error Message

The job details show:

05/28/2013 14:23:50 - Info bpbrm (pid=14698) starting bpbkar on client
05/28/2013 14:23:50 - connected; connect time: 0:00:00
05/28/2013 14:23:52 - Info bpbkar (pid=3672) Backup started
05/28/2013 14:23:52 - Info bpbrm (pid=14698) bptm pid: 14700
05/28/2013 14:23:52 - Info bptm (pid=14700) start
05/28/2013 14:23:52 - Info bptm (pid=14700) using 524288 data buffer size
05/28/2013 14:23:52 - Info bptm (pid=14700) using 16 data buffers
05/28/2013 14:23:52 - Info bptm (pid=14700) USING 524288 data buffer size for FT
05/28/2013 14:24:13 - Error bptm (pid=14700) Could not open FT Server pipe: pipe open failed (589841)
05/28/2013 14:24:13 - Info bptm (pid=14700) EXITING with status 83 <----------
05/28/2013 14:24:16 - Info bpbkar (pid=3672) done. status: 83: media open error
05/28/2013 14:24:16 - end writing
media open error  (83)

Cause

The job details indicate that the bptm process returns the error "could not open FT server pipe." This suggests the error occurred with the FT process on the media server. Reviewing VxUL logging for OID 199 (nbftsrvr) will show additional information about the memory allocation failure.

05/28/13 14:23:53.022 [::OpenPipe] Pre  Pipe=1, freecount=322, blkcount=322
05/28/13 14:23:53.022 [ShmData::OpenBase] shmat(1048590) returned 0xab742000
05/28/13 14:23:53.022 [FTServerShm::AllocDMABuffers] Got index = 0
05/28/13 14:23:53.022 [DMABuffLock] SGRVpool max pages initialized too small, mapvectlen (66) >= maxvectlen (65)
05/28/13 14:23:53.022 [SCPAL_DMABuffAllocE] SCPAL_DMABuffLock of ShmData::LockDMABuffers failure 0x0, sglist =0x2aaaab3305
68
05/28/13 14:23:53.022 [DMABuffLock] SGRVpool max pages initialized too small, mapvectlen (66) >= maxvectlen (65)
05/28/13 14:23:53.022 [SCPAL_DMABuffAllocE] SCPAL_DMABuffLock failure, sglist =0x2aaaab330568
05/28/13 14:23:53.022 [DebugBreak] from scpaldma.c:1838
05/28/13 14:23:53.022 [ShmData::LockDMABuffers] BUFControl[1] SCPAL_DMABuffAlloc segment 0 failure for 524288 bytes at 0xa
b742000, remaining retries 4
05/28/13 14:23:58.013 [DMABuffLock] SGRVpool max pages initialized too small, mapvectlen (66) >= maxvectlen (65)
05/28/13 14:23:58.014 [SCPAL_DMABuffAllocE] SCPAL_DMABuffLock of ShmData::LockDMABuffers failure 0x0, sglist =0x2aaaab330c
38
05/28/13 14:23:58.014 [DMABuffLock] SGRVpool max pages initialized too small, mapvectlen (66) >= maxvectlen (65)
05/28/13 14:23:58.014 [SCPAL_DMABuffAllocE] SCPAL_DMABuffLock failure, sglist =0x2aaaab330c38
05/28/13 14:23:58.014 [DebugBreak] from scpaldma.c:1838
05/28/13 14:23:58.014 [ShmData::LockDMABuffers] BUFControl[1] SCPAL_DMABuffAlloc segment 0 failure for 524288 bytes at 0xa
b742000, remaining retries 3

The retries then count down to 0 where the open pipe request is failed:

05/28/13 14:24:13.042 [ShmData::LockDMABuffers] BUFControl[1] SCPAL_DMABuffAlloc segment 0 failure for 524288 bytes at 0xa
b742000, remaining retries 0
05/28/13 14:24:13.042 [ShmData::LockDMABuffers] SCPAL_DMABuffAlloc failed, calling DeallocDMABuffers
05/28/13 14:24:13.042 [ShmData::DeallocDMABuffers] pDMABuffers = 0x87df30, numBufSeg = 16
05/28/13 14:24:13.042 [FTServer::OpenPipe] (1) BUFControl at 0x87de10 DMA allocation failure, retVal = 589841 = 0x0x90011
05/28/13 14:24:13.042 [::OpenPipe] Failure 0x0x90011 for Pipe ID = 1, direction = 1, shmId = 1048590, shmCtrlId = -1, shmS
ize = -1, numBufSeg = 16, bufSegSize = 524288, bMPX = 0, owner = 0, ownerPid = 14700, backupId = rubik-win_1369765428, ins
tance = 1369765428, No MPX headers - closing shm

Solution

From the job details, a data buffer size of 512K is being allocated for this FT pipe. The allocation for this size is failing due to being too large. The default size for FT data buffers should be 256K. In this case, the touch file below with a value of 262144 resolved the failing FT pipe open.

echo "262144" > /usr/openv/netbackup/db/config/SIZE_DATA_BUFFERS_FT

Applies To

NetBackup 5220 Appliance (configured as FT media server) @ APP_CODE 2.5.2

* As this is a memory allocation issue with data buffers, this error is not NBU-version specific.