How the VERITAS NetBackup (tm) client process handles named pipes (also known as FIFO files)

How the VERITAS NetBackup (tm) client process handles named pipes (also known as FIFO files)

Article: 100023315
Last Published: 2020-10-05
Ratings: 0 1
Product(s): NetBackup

Problem

How the VERITAS NetBackup (tm) client process handles named pipes (also known as FIFO files)

Solution

A named pipe is a special type of device file that can have a script or executable binary writing information into the pipe.
When running cat, more, dd, or any other type of utility on this type of file, that utility will read whatever is being written into the pipe..
 
For NetBackup releases before 7.6, NetBackup's client backup process, bpbkar, operates the same way when explicitly accessing this type of file . When it scans the file, the file descriptor is opened and read via standard system calls, such as open() and read().  If a process has already opened the other end, bpbkar will read and backup whatever the other process is writing to the pipe, until the other process is done. If no other process is writing to the pipe, bpbkar waits. What eventually gets written to the storage unit is what the third-party process is writing to the pipe, not the named pipe itself.

Similarly, when you restore this named pipe, you won't get the named pipe, but rather whatever the output of that named pipe was when it was backed up.  You must manually recreate the named pipe before restoring to it.

Note: Use the raw (-r) option to list or restore the backup images that were created by reading the content from the pipe. That option allows the NetBackup client restore process, tar, to write the previously saved image into the pipe from which it can then be read by a third-party process.  If no process is reading from the pipe, tar waits.
 
The behavior above occurs only if the backup selections include the pathname to the named pipe.  If the backup selection is a parent directory, then the meta-data for the named pipe itself, not the contents read from the pipe, is backed up and restored.

In NetBackup 7.6 – 7.6.1.1, the behavior was changed (E-Track 2312867). To prevent backup hangs when wild-card expansion inadvertently added the pathname of a pipe into the backup selection list, NetBackup was modified to only backup and restore the file for the named pipe, and did not read from or write to the pipe.  This allowed NetBackup to protect the named pipe file, and avoid hanging until timeout when accessing a pipe that had no third-party process writing to (backup) or reading from (restore) the file.

Starting in NetBackup 7.6.1.2, both behaviors are supported (E-Track 3713104). At the request of pre-7.6 customers, NetBackup was modified restore the prior behavior. In order for bpbkar to read the contents from the named pipe, BACKUP_FIFO_FILES = 1 must be added to the bp.conf file on the client host and the pathname to the named pipe must be a backup selection.  If the backup selection is a directory which contains a named pipe file, that behavior remained unchanged.  Only the meta-data for the pipe is backed up, not the contents of the pipe. At restore time, tar will restore/write either the pipe or the contents of the pipe, depending on how the backup was performed.
 
Starting in NetBackup 8.0, the default is changed, but both behaviors are still supported (E-Track 3872746). At the request of customers who preferred all of the pre-7.6 behaviors, the default BACKUP_FIFO_FILES value introduced in NetBackup 7.6.1.2 was flipped with respect to backup selections that specify a named pipe.  The default behavior is to backup the contents of the pipe.  The behavior for backing up a directory containing a named pipe is unchanged, only the meta-data for the pipe file is backed up and restored.  If a named pipe is specified in the backup selections, using BACKUP_FIFO_FILES = 0 causes only the meta-data about the pipe to be backed up, not the contents of the pipe.

The NetBackup pre-7.6 and post-7.6.1.1 behavior regarding named pipes is useful if you need an ad-hoc solution for backing up a hot or running database. Since most commercial databases have freeze, dump, and resume command-line utilities, some administrators have set up named pipes, then scripted solutions that halt or quiesce the database, run its dump command to send a snapshot of the database to a device or file such as a named pipe, and then resume the database operations after the completion of the dump. Using a named pipe allows the backup image to go directly between the database and NetBackup without having to be spooled to an extra/spare disk volume.

There is a caveats with this solution. The backup of the named pipe file contents must be completed within the CLIENT_READ_TIMEOUT bp.conf setting on the NetBackup media server. Otherwise, NetBackup will abort the backup with a non-zero status code (typically 13, 41, or 76 ). So, unless the third-party process writing to the pipe in question exits within that timeout period and closes its end of the pipe,  NetBackup will fail the backup job so that server resources can be allocated to other jobs.  The same is true for restores, except that the job will fail with status 2800. In either case, the NetBackup client processes will continue waiting on the pipe until terminated

In most instances, it's easier to back up (and restore) by using bpstart_notify and bpend_notify scripts to start up and shut down any databases on the client and then reading the database files directly from the file system. Better yet, the database extension products available for NetBackup do the same job, and can back the database up while hot and on-line.

 

Was this content helpful?