NetBackup for Microsoft SQL Server backups fail with 'Operating system error 0x8007000e(Not enough storage is available to complete this operation)'

Problem

NetBackup for Microsoft SQL Server backups fail with 'Operating system error 0x8007000e(Not enough storage is available to complete this operation)'

Error Message

10:18:41.674 [4956.4940] <16> CODBCaccess::LogODBCerr: DBMS MSG - ODBC return code <-x>, SQL State <37000>, SQL Message <18210><[Microsoft][SQL Native Client][SQL Server]BackupVirtualDeviceSet::Initialize: Request large buffers failure on backup device 'VNBU0-4956-4940-1248099521'. Operating system error 0x8007000e(Not enough storage is available to complete this operation.).>.


10:18:41.674 [4956.4940] <16> Dbbackrec::PerformNBOperation: ERR - Error found executing <backup database "master" to VIRTUAL_DEVICE='VNBU0-4956-4940-1248099521' with stats = 10, blocksize = 65536, maxtransfersize = 4194304, buffercount = 2>.

 

Cause

When the SQL Server's MemToLeave area becomes badly fragmented, there is insufficient contiguous space to allocate the buffers required for the VDI (Virtual Device Interface) backups.

Solution

Operating system error -2147024888 (0x8007000e) may display when performing a Backup of SQL databases. This doesn't necessarily mean a VLDB (Very large database), but backups for smaller databases (master) can also fail.
 

This error occurs when the SQL Server's MemToLeave area becomes so badly fragmented that there is insufficient contiguous space available to allocate the number of buffers required for the VDI (Virtual Device Interface) backups. The result is that the backup fails until the SQL Server is restarted to clean up the MemToLeave area.

 

Although reducing the value of the MAXTRANSFERSIZE and NUMBUFS parameters often allows the backup process to complete, this can cause overall performance to decrease. Since the buffers are allocated in memory, less memory is used when the value of the MAXTRANSFERSIZE parameter is reduced.


Microsoft suggests increasing the MemToLeave area for the SQL Server by using the "-g" switch when starting the SQL Server. However, this may just increase the amount of time that it takes for the memory to become fragmented.

Observation indicates that the following factors seem to make this error occur more quickly:

* large amount of data backed up and/or high backup frequency
* default value for the MAXTRANSFERSIZE parameter (6) in the batch file
* using more than one stripe (NUMBUFS) to perform the SQL backup