InfoScale™ 9.0 Storage Foundation and High Availability Configuration and Upgrade Guide - Linux
- Section I. Introduction to SFHA
- Section II. Configuration of SFHA
- Preparing to configure
- Preparing to configure SFHA clusters for data integrity
- About planning to configure I/O fencing
- Setting up the CP server
- Planning your CP server setup
- Installing the CP server using the installer
- Configuring the CP server cluster in secure mode
- Setting up shared storage for the CP server database
- Configuring the CP server using the installer program
- Configuring the CP server manually
- Configuring CP server using response files
- Verifying the CP server configuration
- Configuring SFHA
- Configuring Storage Foundation High Availability using the installer
- Overview of tasks to configure SFHA using the product installer
- Required information for configuring Storage Foundation and High Availability Solutions
- Starting the software configuration
- Specifying systems for configuration
- Configuring the cluster name
- Configuring private heartbeat links
- Configuring the virtual IP of the cluster
- Configuring SFHA in secure mode
- Configuring a secure cluster node by node
- Adding VCS users
- Configuring SMTP email notification
- Configuring SNMP trap notification
- Configuring global clusters
- Completing the SFHA configuration
- About the License Audit Tool
- Verifying and updating licenses on the system
- Configuring SFDB
- Configuring Storage Foundation High Availability using the installer
- Configuring SFHA clusters for data integrity
- Setting up disk-based I/O fencing using installer
- Setting up server-based I/O fencing using installer
- Setting up non-SCSI-3 I/O fencing in virtual environments using installer
- Setting up majority-based I/O fencing using installer
- Enabling or disabling the preferred fencing policy
- Manually configuring SFHA clusters for data integrity
- Setting up disk-based I/O fencing manually
- Setting up server-based I/O fencing manually
- Preparing the CP servers manually for use by the SFHA cluster
- Generating the client key and certificates manually on the client nodes
- Configuring server-based fencing on the SFHA cluster manually
- Configuring CoordPoint agent to monitor coordination points
- Verifying server-based I/O fencing configuration
- Setting up non-SCSI-3 fencing in virtual environments manually
- Setting up majority-based I/O fencing manually
- Performing an automated SFHA configuration using response files
- Performing an automated I/O fencing configuration using response files
- Configuring I/O fencing using response files
- Response file variables to configure disk-based I/O fencing
- Sample response file for configuring disk-based I/O fencing
- Response file variables to configure server-based I/O fencing
- Sample response file for configuring server-based I/O fencing
- Response file variables to configure non-SCSI-3 I/O fencing
- Sample response file for configuring non-SCSI-3 I/O fencing
- Response file variables to configure majority-based I/O fencing
- Sample response file for configuring majority-based I/O fencing
- Section III. Upgrade of SFHA
- Planning to upgrade SFHA
- About the upgrade
- Supported upgrade paths
- Considerations for upgrading SFHA to 9.0 on systems configured with an Oracle resource
- Preparing to upgrade SFHA
- Considerations for upgrading REST server
- Using Install Bundles to simultaneously install or upgrade full releases (base, maintenance, rolling patch), and individual patches
- Upgrading Storage Foundation and High Availability
- Performing a rolling upgrade of SFHA
- Performing a phased upgrade of SFHA
- About phased upgrade
- Performing a phased upgrade using the product installer
- Moving the service groups to the second subcluster
- Upgrading the operating system on the first subcluster
- Upgrading the first subcluster
- Preparing the second subcluster
- Activating the first subcluster
- Upgrading the operating system on the second subcluster
- Upgrading the second subcluster
- Finishing the phased upgrade
- Performing an automated SFHA upgrade using response files
- Upgrading SFHA using YUM
- Performing post-upgrade tasks
- Optional configuration steps
- Re-joining the backup boot disk group into the current disk group
- Reverting to the backup boot disk group after an unsuccessful upgrade
- Recovering VVR if automatic upgrade fails
- Post-upgrade tasks when VCS agents for VVR are configured
- Resetting DAS disk names to include host name in FSS environments
- Upgrading disk layout versions
- Upgrading VxVM disk group versions
- Updating variables
- Setting the default disk group
- About enabling LDAP authentication for clusters that run in secure mode
- Verifying the Storage Foundation and High Availability upgrade
- Planning to upgrade SFHA
- Section IV. Post-installation tasks
- Section V. Adding and removing nodes
- Adding a node to SFHA clusters
- About adding a node to a cluster
- Before adding a node to a cluster
- Adding a node to a cluster using the Veritas InfoScale installer
- Adding the node to a cluster manually
- Adding a node using response files
- Configuring server-based fencing on the new node
- After adding the new node
- Adding nodes to a cluster that is using authentication for SFDB tools
- Updating the Storage Foundation for Databases (SFDB) repository after adding a node
- Removing a node from SFHA clusters
- Removing a node from a SFHA cluster
- Verifying the status of nodes and service groups
- Deleting the departing node from SFHA configuration
- Modifying configuration files on each remaining node
- Removing the node configuration from the CP server
- Removing security credentials from the leaving node
- Unloading LLT and GAB and removing InfoScale Availability or Enterprise on the departing node
- Updating the Storage Foundation for Databases (SFDB) repository after removing a node
- Removing a node from a SFHA cluster
- Adding a node to SFHA clusters
- Section VI. Configuration and upgrade reference
- Appendix A. Installation scripts
- Appendix B. SFHA services and ports
- Appendix C. Configuration files
- Appendix D. Configuring the secure shell or the remote shell for communications
- About configuring secure shell or remote shell communication modes before installing products
- Manually configuring passwordless ssh
- Setting up ssh and rsh connection using the installer -comsetup command
- Setting up ssh and rsh connection using the pwdutil.pl utility
- Restarting the ssh session
- Enabling rsh for Linux
- Appendix E. Sample SFHA cluster setup diagrams for CP server-based I/O fencing
- Appendix F. Configuring LLT over UDP
- Using the UDP layer for LLT
- Manually configuring LLT over UDP using IPv4
- Broadcast address in the /etc/llttab file
- The link command in the /etc/llttab file
- The set-addr command in the /etc/llttab file
- Selecting UDP ports
- Configuring the netmask for LLT
- Configuring the broadcast address for LLT
- Sample configuration: direct-attached links
- Sample configuration: links crossing IP routers
- Using the UDP layer of IPv6 for LLT
- Manually configuring LLT over UDP using IPv6
- About configuring LLT over UDP multiport
- Appendix G. Using LLT over RDMA
- Using LLT over RDMA
- About RDMA over RoCE or InfiniBand networks in a clustering environment
- How LLT supports RDMA capability for faster interconnects between applications
- Using LLT over RDMA: supported use cases
- Configuring LLT over RDMA
- Choosing supported hardware for LLT over RDMA
- Installing RDMA, InfiniBand or Ethernet drivers and utilities
- Configuring RDMA over an Ethernet network
- Configuring RDMA over an InfiniBand network
- Tuning system performance
- Manually configuring LLT over RDMA
- LLT over RDMA sample /etc/llttab
- Verifying LLT configuration
- Troubleshooting LLT over RDMA
- IP addresses associated to the RDMA NICs do not automatically plumb on node restart
- Ping test fails for the IP addresses configured over InfiniBand interfaces
- After a node restart, by default the Mellanox card with Virtual Protocol Interconnect (VPI) gets configured in InfiniBand mode
- The LLT module fails to start
Setting up non-SCSI-3 fencing in virtual environments manually
To manually set up I/O fencing in a non-SCSI-3 PR compliant setup
- Configure I/O fencing either in majority-based fencing mode with no coordination points or in server-based fencing mode only with CP servers as coordination points.
- Make sure that the SFHA cluster is online and check that the fencing mode is customized mode or majority mode.
# vxfenadm -d
- Make sure that the cluster attribute UseFence is set to SCSI-3.
# haclus -value UseFence
- On each node, edit the /etc/vxenviron file as follows:
data_disk_fencing=off
On each node, edit the /etc/sysconfig/vxfen file as follows:
vxfen_vxfnd_tmt=25
- On each node, edit the /etc/vxfenmode file as follows:
loser_exit_delay=55 vxfen_script_timeout=25
Refer to the sample /etc/vxfenmode file.
On each node, set the value of the LLT sendhbcap timer parameter value as follows:
Run the following command:
lltconfig -T sendhbcap:3000
Add the following line to the /etc/llttab file so that the changes remain persistent after any reboot:
set-timer senhbcap:3000
On any one node, edit the VCS configuration file as follows:
Make the VCS configuration file writable:
# haconf -makerw
For each resource of the type DiskGroup, set the value of the MonitorReservation attribute to 0 and the value of the Reservation attribute to NONE.
# hares -modify <dg_resource> MonitorReservation 0
# hares -modify <dg_resource> Reservation "NONE"
Run the following command to verify the value:
# hares -list Type=DiskGroup MonitorReservation!=0
# hares -list Type=DiskGroup Reservation!="NONE"
The command should not list any resources.
Modify the default value of the Reservation attribute at type-level.
# haattr -default DiskGroup Reservation "NONE"
Make the VCS configuration file read-only
# haconf -dump -makero
- Make sure that the UseFence attribute in the VCS configuration file main.cf is set to SCSI-3.
To make these VxFEN changes take effect, stop and restart VxFEN and the dependent modules
On each node, run the following command to stop VCS:
For systemd environments with supported Linux distributions:
# systemctl stop vcs
For other supported Linux distributions:
# /etc/init.d/vcs stop
After VCS takes all services offline, run the following command to stop VxFEN:
For systemd environments with supported Linux distributions:
# systemctl stop vxfen
For other supported Linux distributions:
# /etc/init.d/vxfen stop
On each node, run the following commands to restart VxFEN and VCS:
For systemd environments with supported Linux distributions:
# systemctl start vxfen
# systemctl start vcs
For other supported Linux distributions:
# /etc/init.d/vxfen start
# /etc/init.d/vcs start