InfoScale™ 9.0 Replication Administrator's Guide - AIX

Tunable parameters for the buffer space on the Secondary

The amount of buffer space available for requests coming in to the Secondary over the network is determined by the VVR tunable, vol_max_nmpool_sz, which defaults to 64 megabytes. VVR allocates separate buffer space for each Secondary RVG, the size of which is equal to the value of the tunable vol_max_nmpool_sz. The buffer space on the Secondary must be large enough to prevent slowing the network transfers excessively.

If the buffer is too large, it can cause problems. When a write arrives at the Secondary, the Secondary sends an acknowledgment to the Primary so that the Primary knows the transfer is complete. When the write is written to the data volume on the Secondary, the Secondary sends another acknowledgment, which tells the Primary that the write can be discarded from the SRL. However, if this second acknowledgment is not sent within one minute, the Primary disconnects the RLINK. The RLINK reconnects immediately but this causes disruption of the network flow and potentially other problems. Thus, the buffer space on the Secondary should be sized in such a way that no write can remain in it for one minute. This size depends on the rate at which the data can be written to the disks, which is dependent on the disks themselves, the I/O buses, the load on the system, and the nature of the writes (random or sequential, small or large).

If the write rate is W megabytes/second, the size of the buffer should be no greater than W * 50 megabytes, that is, 50 seconds' worth of writes.

There are various ways to measure W. If the disks and volume layouts on the Secondary are comparable to those on the Primary and you have I/O statistics from the Primary before replication was implemented, these statistics can serve to arrive at the maximum write rate.

Alternatively, if replication has already been implemented, start by sizing the buffer space on the Secondary to be large enough to avoid timeout and memory errors.

While replication is active at the peak rate, run the following command and make sure there are no memory errors and the number of timeout errors is small:

# vxrlink -g diskgroup -i5 stats rlink_name

Then, run the vxstat command to get the lowest write rate:

# vxstat -g diskgroup -i5

The output looks similar to this:

                    OPERATIONS        BLOCKS        AVG TIME(ms)
TYP NAME          READ     WRITE      READ  WRITE   READ  WRITE

Mon 29 Sep 2003 07:33:07 AM PDT
vol srl1             0      1245        0   1663   0.0   9.0
vol archive          0       750        0    750   0.0   9.0
vol archive-L01      0       384        0    384   0.0   5.9
vol archive-L02      0       366        0    366   0.0  12.1
vol ora02            0       450        0    900   0.0  11.1
vol ora03            0         0        0      0   0.0   0.0
vol ora04            0         0        0      0   0.0   0.0

Mon 29 Sep 2003 07:33:12 AM PDT
vol srl1             0       991        0   1389   0.0  20.1
vol archive          0       495        0    495   0.0  10.1
vol archive-L01      0       256        0    256   0.0   5.9
vol archive-L02      0       239        0    239   0.0  14.4
vol ora02            0       494        0    988   0.0  10.0
vol ora03            0         0        0      0   0.0   0.0
vol ora04            0         0        0      0   0.0   0.0

For each interval, add the numbers in the blocks written column for data volumes, but do not include the SRL. Also, do not include any subvolumes. For example, archive-L01, and archive-L02 are subvolumes of the volume archive. The statistics of the writes to the subvolumes are included in the statistics for the volume archive. You may vary the interval, the total time you run the test, and the number of times you run the test according to your needs. In this example, the interval is 5 seconds and the count is in blocks, hence on a machine with 2 kilobytes of block size, the number of megabytes per interval, M, is (total * 2048)/(1024*1024), where total is the sum for one interval. Hence, for one second the number of megabytes is M/5 and the size of the buffer is (M/5)*50. If there is more than one Primary, do not increase the buffer size beyond this number.

The writes to the SRL should not be considered part of the I/O load of the application. However, in asynchronous mode, the Secondary writes the incoming updates to both the Secondary SRL and the data volumes, so it may be necessary to make the value of vol_max_nmpool_sz slightly larger. However, to avoid the problems discussed at the beginning of this section, the calculated vol_max_nmpool_sz value should still ensure that writes do not remain in the pool for more than one minute.