InfoScale™ 9.0 SmartIO for Solid-State Drives Solutions Guide - Linux

Last Published:
Product(s): InfoScale & Storage Foundation (9.0)
Platform: Linux
  1. Introducing SFHA Solutions SmartIO
    1.  
      About SmartIO for solid-state drives
    2.  
      About SmartIO in an SFHA environment
    3.  
      About SmartIO in an Active/Active cluster environment
    4.  
      About SmartIO in the Solaris virtualized environments
    5.  
      About SmartIO in the Linux virtualized environment
    6.  
      About the SmartIO caching profiler tool
  2. Using the SmartIO feature: use cases
    1. About SmartIO read caching for applications running on VxVM volumes
      1.  
        Required configuration for SmartIO read caching for VxVM volumes
      2.  
        Automatic caching for VxVM volumes
      3.  
        Setting up SmartIO read caching for VxVM volumes
      4.  
        Verifying the VxVM cache area and monitoring the caching
    2. About SmartIO read caching for applications running on VxFS file systems
      1.  
        Required configuration for SmartIO read caching for a VxFS file system
      2.  
        Automatic caching for VxFS file systems
      3.  
        Setting up SmartIO read caching for VxFS file systems
      4.  
        Verifying the VxFS cache area and monitoring the caching
      5.  
        Customizing the caching behavior
    3. About SmartIO caching on SSD devices exported by FSS
      1.  
        Status of cache areas when nodes leave or join the cluster
      2.  
        Setting up cache areas using SSDs exported by FSS
    4. About SmartIO write-back caching for applications running on VxFS file systems- DLV 11 to 13
      1.  
        Required configuration for SmartIO write-back caching for a VxFS file system
      2.  
        Setting up SmartIO write-back caching for VxFS file systems
      3.  
        Verifying the VxFS cache area and monitoring the caching (writeback mode)
    5. About SmartIO FEL-based writeback caching for applications running on VxFS file systems- DLV 14 and later
      1.  
        Required configuration for FEL-based write-back caching for a VxFS file system
      2.  
        Setting up FEL-based write-back caching for VxFS file systems
      3.  
        Verifying the VxFS cache area and monitoring FEL-based write-back caching
    6. About multiple SmartIO cache areas for read and write-back caching on VxFS file systems
      1.  
        About the smartiocache option
      2.  
        Converting VxFS cache areas from one type to another
      3.  
        Setting up multiple cache areas on a system
      4.  
        Verifying the VxFS cache areas
    7. About SmartIO caching for Oracle databases on VxFS file systems
      1.  
        Prerequisites and configuration for using the SmartIO plug-in for Oracle
      2.  
        Setting default SmartIO caching policies for a database running on a VxFS file system
      3.  
        Setting SmartIO caching policies for database objects
      4.  
        Pinning and unpinning database objects
      5.  
        Enabling and disabling caching for the database
      6.  
        Listing cache policy details for the database
      7.  
        Listing cache statistics for the database
    8. About SmartIO caching for databases on VxVM volumes
      1.  
        Applying a SmartIO database caching template for a VxVM volume
    9.  
      Technology Preview: Distributed SmartIO in Veritas InfoScale storage environments
  3. Administering SmartIO
    1.  
      Creating a cache area
    2.  
      Displaying information about a cache area
    3. Enabling or disabling caching for a data object
      1.  
        Enabling or disabling caching for a file system
      2.  
        Enabling or disabling caching for a data volume
    4.  
      Adding a device to the cache area
    5.  
      Pausing caching from a volume to a cache area
    6.  
      Removing a device from the cache area
    7.  
      Destroying a cache area
    8.  
      Setting the attributes of the VxVM cache area
    9.  
      Setting or changing the caching mode for a VxFS cache area
    10.  
      Flushing dirty data from a writeback cache area
    11.  
      Tuning the writeback caching
    12. Viewing the SmartIO cache statistics
      1.  
        Viewing the detailed caching stats for a VxVM cache area
      2.  
        Viewing the detailed caching stats for a VxFS cache area
  4. Troubleshooting and error handling
    1. Support for a persistent or 'warm' VxVM cache
      1.  
        Primary volume failure with a stale cache could cause possible data corruption
      2.  
        Migrating a cache during HA failover is not supported
    2.  
      Cache area is lost after a disk failure (3158482)
    3.  
      Cache is not online after a reboot
    4.  
      Recovering the write-back cache after a node failure
  5. Appendix A. Command reference
    1.  
      SmartIO command reference

Technology Preview: Distributed SmartIO in Veritas InfoScale storage environments

Distributed SmartIO is a new feature available as a technology preview in Veritas InfoScale for configuration and testing in non-production environments. It is primarily targeted for Oracle RAC or ODM.

Note:

All the configurations which are supported on SmartIO read caching for applications running on VxFS file systems are also supported with Distributed SmartIO.

With the advancements in the hardware technology - network interconnect such as infiniband, accessing and sharing the data using the network rather than the disk as a medium for data sharing is proving to be faster and cost efficient in storage environments. Data can be cached on faster but costlier SSD storage on few nodes in a cluster. High speed network interconnect can be used to fetch the data as required on any node in the cluster.

Considering these benefits, Veritas InfoScale has come up with a robust solution, Distributed SmartIO, which lets you share SSD resources between all the nodes in the cluster for caching frequently read data.

Distributed SmartIO feature uses SSD resources both local and remote for caching purposes to improve performance of the applications running within the cluster. Using Distributed SmartIO, you can create a coherent cache area using SSD across all the nodes in the cluster irrespective of whether each node has an independent local cache.

In the Distributed SmartIO feature, a few or all the nodes in the cluster have SmartIO cache attached to it. Each node within the cluster is connected through a high speed network interconnect such as Infiniband or 10G Ethernet. Read Operation (RO) cached data and metadata about the cached data is distributed and shared across the SmartIO cache within all nodes in the cluster. The metadata also resides at the back-end storage device.

See Read Operations (RO) in Distributed SmartIO

See Write Operations in Distributed SmartIO

Read Operations (RO) in Distributed SmartIO

In a clustered environment, when the application issues a RO using Direct I/O, Distributed SmartIO checks to see if the RO can be serviced from the local cache on the existing node. In the absence of requested data from local cache on the existing node, Distributed SmartIO attempts to serve the RO from remote node's local cache. If the requested data is not present in any cache, then the data is rendered asynchronously in the Distributed SmartIO from the back-end storage.

Note:

You can create a coherent cache area using SSDs on all nodes in the cluster even if one or more nodes within the cluster does not have an independent SmartIO (Local Cache) attached to it.

The following diagram gives an overview of how the read operations are processed in Distributed SmartIO:

Write Operations in Distributed SmartIO

In a clustered environment, when the application issues a write operation on any node within the cluster, it invalidates copies of the data from all the remote caches except the data on local cache of the node on which the write operation is performed. The local cache of this node is updated with the new data.

Note:

You can create a coherent cache area using SSDs on all nodes in the cluster even if one or more nodes within the cluster does not have an independent SmartIO (Local Cache) attached to it.

The following diagram gives an overview of how the write operations are processed in Distributed SmartIO:

Enabling Distributed SmartIO feature in Veritas InfoScale storage environment

To enable Distributed SmartIO feature in Veritas InfoScale storage environment

  1. Use sfcache command to create SmartIO cache on the required nodes in the cluster.

    See Creating a cache area . for more details.

    See sfcache(1M) manual page for more details.

  2. While mounting a VxFS file system, use cluster,smartiomode=cfusion option to enable Distributed SmartIO.
    # mount -t vxfs /dev/vx/dsk/testdg/vol1 -o cluster,
smartiomode=cfusion /mnt1
  3. To see statistics on the cache usage, use the following command: 
    # sfcache stat
       
Cache Name: cache_dg1/cachevol
Cache Size:   24.97 GB
Cache Utilization:    70.8 MB ( 0.28 %)
File Systems Using Cache:       6
Writeback Cache Use Limit:  Unlimited
Writeback Flush Timelag:     10 s

Read Cache                            Writeback
Hit Ratio   Data Read  Data Written   Hit Ratio  Data Written  rdcachename   wbcachename
 
Total:
45.40 %     4.99 MB     2 MB           0.00 %     0 KB

/mnt1:
74.98 %     2.996 MB    1 MB           0.00 %     0 KB        cache_dg1/cachevol     -
Remote Read:
66.62 %     1.996 MB

The output displays the Data Read and Hit Ratio statistics under the Remote Read section for a particular mount point. It provides statistical information about the data that is served by SmartIO cache (Local Cache) for reads happening on the remote nodes.

Limitations of Distributed SmartIO

Support for Distributed SmartIO is limited by the following constraints:

  • Distributed SmartIO feature is supported only for applications running on VxFS file system

  • Distributed SmartIO feature is beneficial only when used with Direct I/O