Veritas InfoScale™ 8.0.2 Replication Administrator's Guide - Linux
- Section I. Getting started with Volume Replicator
- Introducing Volume Replicator
- Understanding how Volume Replicator works
- How VVR processes application writes
- How VVR uses kernel buffers for replication
- How data flows in VVR asynchronous mode
- About secondary logging enabled asynchronous mode
- About bulk transfer with secondary logging
- How data flows in VVR asynchronous mode with secondary logging
- How data flows in VVR synchronous mode
- How data flows in VVR adaptive synchronous mode
- How data flows in an RDS containing multiple Secondary hosts
- Replication in a shared disk group environment
- Assigning a slave node as a logowner
- Understanding how VVR logs writes to the SRL
- Understanding Storage Checkpoints
- Volume sets in VVR
- Changing membership of an RVG and a volume set
- Using SmartTier with VVR
- Cross-platform Data Sharing in VVR
- Understanding the VVR snapshot feature
- About VVR compression
- Understanding data encryption over the wire
- Planning and configuring replication
- Introduction to planning and configuring replication
- Before you begin configuring
- Choosing the mode of volume replication
- Choosing latency and SRL protection
- Planning the network
- Sizing the SRL
- Choosing the type of DCM logging
- Best practices for setting up replication
- How the agents for hybrid applications work
- Pre-requisites to enable data over wire encryption
- Enabling Data Change Object (DCO) for SRL Volumes
- Understanding replication settings for a Secondary
- Configuring VVR in a VCS environment
- Overview of how to configure VVR in a VCS environment
- Configuring VVR with virtual hostname to map with multiple IPs (Stretched cluster support)
- Using the primary-elect feature to choose the primary site after a site disaster or network disruption
- Requirements for configuring VVR in a VCS environment
- Generic VVR setup in a VCS environment
- Example VVR configuration in a VCS environment
- Example RVG configuration for a failover application
- Example RVG configuration for a parallel application
- Example setting up VVR in a VCS environment
- Configuring the agents for a bunker replication configuration
- Administering VCS service groups
- Section II. Setting up and administering VVR
- Setting up replication
- About configuring VVR replication
- Enabling security certificate
- Setting up the certificate authority (CA) certificates in /etc/vx/vvr/cacert.pem
- Creating a Replicated Data Set
- Creating a Primary RVG of an RDS
- Adding a Secondary to an RDS
- Changing the replication settings for a Secondary
- Configuring adaptive synchronous replication
- Synchronizing the Secondary and starting replication
- Starting replication when the data volumes are zero initialized
- Starting replication over the wire when data volumes are encrypted
- Setting up replication with different sector-sized disks
- Displaying configuration information
- Displaying RVG and RDS information
- Displaying information about data volumes and volume sets
- Displaying information about Secondaries
- Displaying a list of Storage Checkpoints
- Displaying statistics with the vrstat display commands
- Displaying the consolidated statistics
- Displaying the RLINK information for all the hosts in the RDS
- Displaying information about all the data volumes for all the hosts in the RDS
- Displaying information about the SRL volumes for all the hosts in the RDS
- Displaying information about the memory tunable parameters for all the hosts in the RDS
- Determining VVR network bandwidth usage and compression ratio
- Collecting consolidated statistics of the VVR components
- Displaying network performance data
- VVR event notification
- Administering Volume Replicator
- Administering data volumes
- Associating a volume to a Replicated Data Set
- Associating a volume set to an RDS
- Associating a Data Change Map to a data volume in DCO
- Associating a Data Change Map to a data volume as a log plex
- Resizing a data volume in a Replicated Data Set
- Renaming a data volume in a Replicated Data Set
- Dissociating a data volume from its Replicated Data Set
- Mapping the name of a Secondary data volume to a differently named Primary data volume
- Mapping disk groups
- Administering the SRL
- Protecting from SRL overflow
- Incrementally synchronizing the Secondary after SRL overflow
- SRL overflow protection with DCM - flags and definitions
- Prerequisite for incrementally synchronizing the Secondary
- Breaking off mirrors before incremental synchronization
- Example - Resynchronizing the Secondary using break off mirrors
- Example - Recreating volumes if a disaster occurs during resynchronization
- Notes on using incremental synchronization on SRL overflow
- Changing the size of the SRL on the Primary and the Secondary
- Decreasing the size of the SRL on the Primary
- Administering replication
- Administering the Replicated Data Set
- Administering Storage Checkpoints
- Creating RVG snapshots
- Using the instant snapshot feature
- About instant full snapshots
- Prerequisites for creating instant full snapshots
- Creating snapshot volumes for data volumes in an RVG
- Preparing the volumes prior to using the instant snapshot feature
- Freezing or pausing replication prior to taking a snapshot
- Creating instant full snapshots
- Unfreezing or resuming replication after taking a snapshot
- About instant space-optimized snapshots
- Preparing the RVG volumes for snapshot operation
- Creating the cache object for instant space-optimized snapshots
- Freezing or pausing replication prior to creating an instant space-optimized snapshot
- Creating instant space-optimized snapshots
- Unfreezing or resuming replication after taking an instant space-optimized snapshot
- About instant plex-breakoff snapshots
- Administering snapshots
- Using the traditional snapshot feature
- Using Veritas Volume Manager FastResync
- Verifying the DR readiness of a VVR setup
- Backing up the Secondary
- Administering data volumes
- Using VVR for off-host processing
- Transferring the Primary role
- About transferring the Primary role
- Migrating the Primary
- About taking over from an original Primary
- Failing back to the original Primary
- About choosing the Primary site after a site disaster or network disruption
- Application availability in the case of a network disruption
- Configuring VCS global clustering so you can choose the Primary site
- Choosing the Primary site after a site disaster or network disruption
- Troubleshooting the primary-elect feature
- Primary-elect configuration limitations
- Replication using a bunker site
- Introduction to replication using a bunker site
- Sample bunker configuration
- Setting up replication using a bunker site
- Administering replication using a bunker site
- Using a bunker for disaster recovery
- Replication using a bunker site in a VCS environment
- Removing a bunker
- About bunker commands
- Troubleshooting VVR
- Recovery from RLINK connect problems
- Recovery from configuration errors
- Errors during an RLINK attach
- Errors during modification of an RVG
- Recovery on the Primary or Secondary
- About recovery from a Primary-host crash
- Recovering from Primary data volume error
- Primary SRL volume error cleanup and restart
- Primary SRL volume error at reboot
- Primary SRL volume overflow recovery
- Primary SRL header error cleanup and recovery
- Secondary data volume error cleanup and recovery
- Secondary SRL volume error cleanup and recovery
- Secondary SRL header error cleanup and recovery
- Secondary SRL header error at reboot
- Tuning replication performance
- Overview of replication tuning
- SRL layout
- Tuning Volume Replicator
- VVR buffer space
- Write buffer space on the Primary
- Readback buffer space on the Primary
- Buffer space on the Secondary
- Tunable parameters for the VVR buffer spaces
- Tunable parameters for the write buffer space on the Primary in a private disk group
- Tunable parameter for the readback buffer space
- Tunable parameters for the buffer space on the Primary in a shared disk group
- Tunable parameters for the buffer space on the Secondary
- DCM replay block size
- Heartbeat timeout
- Memory chunk size
- UDP replication tuning
- Tuning the number of TCP connections
- Message slots on the Secondary
- VVR and network address translation firewall
- Tuning VVR compression
- VVR buffer space
- Setting up replication
- Section III. Getting started with File Replicator
- Introducing File Replicator
- Administering File Replicator
- About vfradmin utility
- Starting the vxfstaskd and vxfsrepld file replication daemons
- Protecting a target file system
- Creating a file replication job
- Creating a consistency group
- Managing a file replication job
- Displaying file replication job information
- Modifying a file replication job
- Modifying a consistency group
- Working with pattern lists
- Deleting a file replication job
- Deleting a consistency group
- Migrating VFR to support IPv6/dual-stack
- Performing a VFR switchover
- Performing a VFR failover after a disaster
- Recovering a failed site if the failed source node comes up again
- Recovering a failed site if a new node is assigned as the target
- Section IV. Analyzing your environment with Volume Replicator Advisor
- Introducing Volume Replicator Advisor (VRAdvisor)
- Collecting the sample of data
- About collecting the sample of data
- Collecting the sample of data on UNIX
- Collecting the sample of data on Windows
- Analyzing the sample of data
- About analyzing the sample of data
- Launching the VRAdvisor wizard
- Analyzing the collected data
- Understanding the results of the analysis
- Viewing the analysis results
- Recalculating the analysis results
- Applying different parameters to the existing sample of data
- Performing What-if analysis
- Calculating the SRL Size for a specified Network Bandwidth and Outage
- Calculating the Network Bandwidth for data loss specified in bytes
- Calculating the Network Bandwidth for data loss specified in time duration
- Calculating the Network Bandwidth for Bunker and RTO
- Changing the value ranges on the slider bar
- Recording and viewing the results
- Installing Volume Replicator Advisor (VRAdvisor)
- Section V. VVR reference
- Appendix A. VVR command reference
- Appendix B. Using the In-band Control Messaging utility vxibc and the IBC programming API
- About the IBC messaging utility vxibc
- In-band Control Messaging overview
- Using the IBC messaging command-line utility
- Registering an application name
- Displaying the registered application name for an RVG
- Receiving an IBC message
- Sending an IBC message
- Unfreezing the Secondary RVG
- Unregistering an application name
- Receiving and processing an IBC message using a single command
- Sending and processing an IBC message using a single command
- Examples - Off-host processing
- Example 1 - Decision support using the traditional snapshot feature and the vxibc utility
- Example 2 - Backing up using the snapshot feature and the vxibc utility
- Example 3 - Trial failover using the snapshot feature
- Example 4 - Decision support using the instant full snapshot feature and the vxibc utility
- In-band Control Messaging API
- Appendix C. Volume Replicator object states
- Appendix D. Alternate methods for synchronizing the Secondary
- Using the full synchronization feature
- Using block-level backup and Storage Checkpoint
- Using the Disk Group Split and Join feature
- Using difference-based synchronization
- Examples for setting up a simple Volume Replicator configuration
- Creating a Replicated Data Set for the examples
- Example for setting up replication using full synchronization
- Example for setting up replication using block-level backup and checkpointing
- Example for setting up replication using Disk Group Split and Join
- Example for setting up replication using differences-based synchronization
- Example for setting up replication when data volumes are initialized with zeroes
- Appendix E. Migrating VVR from IPv4 to IPv6
- Migrating VVR to support IPv6 or dual stack
- Overview of VVR migration from IPv4 to IPv6
- About migrating to IPv6 when VCS global clustering and VVR agents are not configured
- About migrating to IPv6 when VCS global clustering and VVR agents are configured
- Understanding the current IPv4 configuration when VCS global clustering and VVR agents are configured
- Migration prerequisites when VCS global clustering and VVR agents are configured
- Migrating to IPv6 when VCS global clustering and VVR agents are configured
- Migrating the VCS global clustering service group to IPv6 when VCS global clustering and VVR agents are configured
- Adding IP and NIC resources for IPv6 addresses in the RVG agent group when VCS global clustering and VVR agents are configured
- Migrating VVR RLINKs from IPv4 to IPv6 when VCS global clustering and VVR agents are configured
- Removing the IPv4 resources from the VCS configuration when VCS global clustering and VVR agents are configured
- About migrating to IPv6 when VCS global clustering and VVR agents are configured in the presence of a bunker
- Understanding the current IPv4 configuration when VCS global clustering and VVR agents are configured in the presence of a bunker
- Migration prerequisites when VCS global clustering and VVR agents are configured in the presence of a bunker
- Migrating to IPv6 when VCS global clustering and VVR agents are configured in the presence of a bunker
- Migrating the VCS global clustering service group to IPv6 when VCS global clustering and VVR agents are configured in the presence of a bunker
- Adding the IP and NIC resources for IPv6 addresses in the RVG agent group when VCS global clustering and VVR agents are configured in the presence of a bunker
- Migrating VVR RLINKs from IPv4 to IPv6 when VCS global clustering and VVR agents are configured in the presence of a bunker
- Removing the IPv4 resources from the VCS configuration when VCS global clustering and VVR agents are configured in the presence of a bunker
- Appendix F. Sample main.cf files
Recovery from RLINK connect problems
This section describes the errors that may be encountered when connecting RLINKs. To be able to troubleshoot RLINK connect problems, it is important to understand the RLINK connection process.
Connecting the Primary and Secondary RLINKs is a two-step operation. The first step, which attaches the RLINK, is performed by issuing the vradmin startrep command. The second step, which connects the RLINKs, is performed by the kernels on the Primary and Secondary hosts.
When the vradmin startrep command is issued, VVR performs a number of checks to ensure that the operation is likely to succeed, and if it does, the command changes the state of the RLINKs from DETACHED/STALE to ENABLED/ACTIVE. The command then returns success.
If the command is successful, the kernel on the Primary is notified that the RLINK is enabled and it begins to send messages to the Secondary requesting it to connect. Under normal circumstances, the Secondary receives this message and connects. The state of the RLINKs then changes from ENABLED/ACTIVE to CONNECT/ACTIVE.
If the RLINK does not change to the CONNECT/ACTIVE state within a short time, there is a problem preventing the connection. This section describes a number of possible causes. An error message indicating the problem may be displayed on the console.
If the following error displays on the console:
VxVM VVR vxrlink INFO V-5-1-5298 Unable to establish connection with remote host <remote_host>, retrying
Make sure that the vradmind daemon is running on the Primary and the Secondary hosts; otherwise, start the vradmind daemon by issuing the following command:
For systemd environments with supported Linux distributions:
# systemctl start vras-vradmind.sh
For other supported Linux distributions:
# /etc/init.d/vras-vradmind.sh start
For an RLINK in a shared disk group, make sure that the virtual IP address of the RLINK is enabled on the logowner.
If there is no self-explanatory error message, issue the following command on both the Primary and Secondary hosts:
# vxprint -g diskgroup -l rlink_name
In the output, check the following:
The remote_host of each host is the same as local_host of the other host.
The remote_dg of each host is the same as the disk group of the RVG on the other host.
The remote_dg_dgid of each host is the same as the dgid (disk group ID) of the RVG on the other host as displayed in the output of the vxprint -l diskgroup command.
The remote_rlink of each host is the same as the name of the corresponding RLINK on the other host.
The remote_rlink_rid of each host is the same as the rid of the corresponding RLINK on the other host.
Make sure that the network is working as expected. Network problems might affect VVR, such as prevention of RLINKs from connecting or low performance. Possible problems could be high latency, low bandwidth, high collision counts, and excessive dropped packets.
For an RLINK in a private disk group, issue the following command on each host.
For an RLINK in a shared disk group, use vxprint -Vl | grep logowner to find the logowner node, then issue the following command on the logowner on the Primary and Secondary.
# ping remote_host
Note:
This command is only valid when ICMP ping is allowed between the VVR Primary and the VVR Secondary.
After 10 iterations, type Ctrl-C. There should be no packet loss or very little packet loss. To ensure that the network can transmit large packets, issue the following command on each host for an RLINK in a private disk group.
For an RLINK in a shared disk group, issue the following command on the logowner on the Primary and Secondary:
# ping -s 8192 remote_host
The packet loss should be about the same as for the earlier ping command.
Issue the vxiod command on each host to ensure that there are active I/O daemons. If the output is 0 volume I/O daemons running, activate I/O daemons by issuing the following command:
# vxiod set 10
VVR uses well-known ports to establish communications with other hosts.
Issue the following command to display the port number:
# vxprint -g diskgroup -l rlink_name
Issue the following command to ensure that the heartbeat port number in the output matches the port displayed by vxprint command:
# vrport
Confirm that the heartbeat port has been opened by issuing the following command:
# netstat -an | grep port-number
where port-number is the port number being used by the heartbeat server as displayed by the vrport command.
The output looks similar to this:
udp 0 0 0,0,0,0:port-number
Check for VVR ports on the Primary and Secondary sites.
Run the vrport utility and verify that ports are same at both ends.
Check whether the required VVR ports are open. Check for UDP 4145, TCP 4145, TCP 8199, and the anonymous port. Enter the following commands:
# netstat -an --udp | grep 4145 udp 0 0 :::4145 :::* # netstat -an --tcp | grep 4145 tcp 0 0 :::4145 :::* LISTEN # netstat -an --tcp | grep 8199 tcp 0 0 :::8199 :::* LISTEN
Perform a telnet test to check for open ports. For example, to determine if port 4145 is open, enter the following:
# telnet <remote> 4145
Use the netstat command to check if vradmind daemons can connect between the Primary site and the Secondary site:
# netstat -an --tcp | grep 8199 | grep ESTABLISHED tcp 0 0 10.209.85.18:56872 10.209.85.19:8199 ESTABLISHED tcp 0 0 10.209.87.175:8199 10.209.85.23:42860 ESTABLISHED tcp 0 0 10.209.87.175:8199 10.209.87.176:9335 ESTABLISHED