Recommended Practices - Dedupe to Cloud Storage with Backup Exec Open Dedupe

Recommended Practices - Dedupe to Cloud Storage with Backup Exec Open Dedupe

Article: 100041148
Last Published: 2018-01-09
Ratings: 0 0
Product(s): Backup Exec

Description

Supportability and Requirements

Backup Exec supports dedupe to cloud for Google, Azure, Amazon and Local Storage from version 16 Feature Pack

Backup Exec Deduplication Option/Feature is required to perform Dedupe to Cloud with Open Dedupe.

Local Disk Storage requirements for Open Dedupe to Cloud -
                         a.     File Metadata - 2.1 percent of non Deduplicated data
                         b.     Hash Table - 0.4 percent of unique storage size
                         c.     Local Cache - By default set to 100 GB. Refer step 3c.

Local Disk Storage requirement for Open Dedupe to Local Storage -

                         a.     Local Unique Data Store -  OpenDedupe stores all unique blocks locally for volumes not backed by object storage. For Object storage backed volumes, this is not used. Local storage size depends on the data being backed up and retention, but typically represents 100% of the front end data for a 60 Day retention. OpenDedupe uses a similar variable block deduplication method

4.     Local Storage performance requirement -
                   - 2000 random read IOPS
                   - 2400 random write IOPS
                   - 180 MB/s of streaming reads
                   - 120 MB/s of streaming writes

5.     CPU Size - For high dedupe primary limiter for Open Dedupe performance will be the CPU.  For a single 16 Core CPU, SDFS i.e. Open Dedupe performs at
                   - 2GB/s for 2% Unique Data
                   - Speed of local disk for 100% unique data. Using minimum requirements this would equal 120MB/s.


There are 4 stages involved in configuring Backup to Open Dedupe Storage on Cloud or Local.
Refer the Install and Config guide for more details. In summary the steps are as follows:

Step 1 - Configure the Storage (Bucket/Container) in the Cloud or allocate a local disk if configuring open dedupe on Local Disk.

           - Create a Cloud container or bucket and make sure to note the Access Key.

Step 2 - Download and Install the Open Dedupe Software

           - Download the SDFS and OST binaries from under the Backup Exec section http://opendedup.org/odd/download/

           - Install SDFS on the drive which is not a system drive. This helps to yield better performance.

Note: SDFS is the Open Dedupe software that will do the deduplication before sending data to cloud storage

           -Install Microsoft Visual C++ before Launching OpenDedupeConfigUI.exe


Step 3 - Configure the Cloud Bucket/Container or Local Volume using OpenDedupeConfigUI.exe -

                       a.   Download the Tool
            
                       b.   Launch OpenDedupeConfigUI.exe and fill the Volume (LSU) Name.
Note: This LSU name will need to be selected from within Backup Exec Configure Storage when configuring Open Dedupe as an OST in Backup Exec.

                       c.     Set the Cached data path to a dedicated drive other than the system partition.
                       d.     The Local Cache Size should be set as the maximum restore size. This helps with Backup Restore performance.
                       e.     The Open Dedupe Configuration Password set in OpenDedupeConfigUI.exe should be noted as it will be needed later while configuring the Storage within Backup exec. This password will be used to configure the account that will be used to access this storage as an OST.

                       f.     Ensure container, bucket name etc. along with the Access Key , Storage account Name are filled in accurately.

                       g.    The OST server name can be anything other than the BE server name. This server name will be required when configuring OST from within Backup Exec Storage configuration wizard.

                       e.     The OpenDedupeConfigUI.exe can be used to configure more Volume LSUs. Multiple Open Dedupe Storage can be configured on the same Backup Exec Server.

                       f.     The user account used to configure Open Dedupe  can have any username but the password need to be the one set in OpenDedupeConfigUI.exe. This account should be configured in Backup Exec before configuring the Open dedupe storage as an OST in Backup Exec.

Step 4 - Configure Storage in Backup Exec

                       a.     From Backup Exec Storage Tab -> Click on Configure Storage
                       b.     Select Network Storage and click Next
                       c.     Choose Open Storage and click Next
                       d.     Provide the Name of the Device. Any name can be chosen. Then click Next
                       e.     Choose OpenDedupe provider and click Next
                       f.     Provide the ServerName and Logon Account of Open Dedupe. This is name Server name given in the OpenDedupeConfiguration Tool. As mentioned earlier the account name can be anything but password has to match with the one provided in OpenDedupeConfiguration tool. Click Next
                       g.     Choose the LSU and then click Next. Configure concurrent operation setting, click next and finish the configure. Restart BE services and Open Dedupe Storage is configured for use in BE.

Step 5 - Create a Backup Job. If Open dedupe is configured on Cloud Bucket/Container, then disable Verify on the Backup Job.

Additional Important Notes:

Resource that support GRT Technology can be backed up to Open Dedupe Storage. Staging is required for restoring an individual item from a GRT enabled backup set. It is recommended to have space as large as the backup set as it is needed during GRT restore.

The Backup Sets expire as per the retention set during the Backup and according to DLM rules. To check the Storage capacity run the following command from the path where SDFS is installed , Run command prompt as administrator -
Sdfscli.exe -nossl -volume-info -password password [-- port ]

 If you have deleted some backup sets, you wait for periodic garbage collection to free storage or you can force immediate garbage collection with the following command:

sdfscli --nossl --password <password> --cleanstore

Opt Dupe can be performed between 2 Open Dedupe Storage Devices. It is recommended not to share the Open Dedupe device in a CAS MBES environment.

Refer 000127160 for more details on BE interaction with Open Dedupe components during Backup and Restore Operation.

Refer 100040672 Cloud OpenDedupe store DR to alternative Backup Exec server

Was this content helpful?