Problem
When comparing the byte count and job duration of differential and incremental backup jobs of VMware virtual machines on ESX 4.0, the figures indicate that a full backup has been run.
A review within the vSphere Client of the summary information for the virtual machines does show that they are configured to use VM Hardware Version 7 and so meets the required level. However, if the Changed Block Tracking settings within each virtual machine are not set, then a full backup is the result.
Cause
Backup Exec 2010 has been designed to create or modify these settings without needing to shutdown your virtual machines and attempts to make the required changes at the start of the Full Backup job. The settings will remain in place once they are correctly set. In some environments the request by Backup Exec to modify the settings fails and causes a full backup to be run even when a differential or incremental backup is selected.
It is possible to review the ctkEnabled settings inside the VMX files of the virtual machines (without shutting them down). In order to review the Changed Block Tracking Settings inside the vSphere Client:
1) Make sure the policy based full backup job of the Virtual Machine has run at least once (there will correctly be no settings if this has not happened)
2) Shutdown the virtual machine
3) In the vSphere Client, edit the settings of the virtual machine
4) Click on the Options tab
5) Under Advanced section select General
6) Click the Configuration Parameters button - this will be grayed out if the virtual machine is running.
7) Confirm if ctkEnabled and one or more scsiN:N.ctkEnabled entries exist and are set to true, as can be seen in Figure 1 below.
NOTE: there should be one scsiN:N.ctkEnabled (where N:N represents the scsi device identifiers) per virtual Hard Disk
Fig 1. Correct Changed Block Tracking Settings
* If these settings are not created and set to true by the initial backup, then all jobs will run as a full backup.
** Investigation has shown that this can be caused by the credentials in the Backup Selection List being linked with an ESX security role that does not have enough permissions to modify the changed tracking settings. In particular the role called "VMware Consolidated Backup user (sample)" does not have enough permissions.
Solution
Configure the Windows based user (if using a Virtual Center) or the ESX/Linux user (if using direct to ESX Host) so it is linked to an ESX security role that has enough permissions. The specific requirements for AVVI Backups are discussed in the article provided at the bottom of this document.
* Note: The only pre-defined security role that has enough permissions to backup and restore is the Administrator role.
To confirm which User or Group of users is assigned to a specific Virtual Machine and which role they are assigned, review the permissions screen/tab of the Virtual Machine itself inside the vSphere Client. Additionally, this screen also shows what level the permissions are defined as VMware security does have a concept of security inheritance (propagation).
To review the user and role assignments applied to each datastore (which can also affect the backup and restore processes), inside the vSphere Client, select View --> Inventory --> Datastores and then select the required datastore and click the Permissions tab.
To review the specific permissions applied to each role, in the vSphere Client select View --> Administration --> Roles from the menu. Then right click the role of interest and select "Edit Role..." Note: the built in roles of Read-only and Administrator cannot be edited, but can be cloned to either review the settings or base a custom role around the settings.
In practice, once the correct security is configured, most systems will then perform incremental and differential backups. However some virtual machines may require a one-time reboot due to the running state of the system still denying the ability to enable the Changed Tracking settings. Veritas cannot predict which virtual machines might need a restart.
Workaround:
Should the above solution fail to allow differential and incremental backups to run correctly, then the required ctkEnabled settings can be manually created or modified in the Configuration Parameters screen as a workaround:
1) Make sure the virtual machine is shutdown
2) In the vSphere client, edit the settings of the virtual machine
3) Click on the first Hard Disk and make a note of the SCSI (N:N) reference listed against the Virtual Device Node
4) Repeat step 2 for each disk
5) Click on the Options Tab
6) Under Advanced section select General
7) Click the Configuration Parameters button - this will be grayed out if the virtual machine is running.
8) Use the Add Row button to create a row called ctkEnabled and set to true
9) Add an additional row for the first Hard Disk called scsiN:N.ctkEnabled (where the N:N in the row name matches the information recorded in step 2) and set the value to true
10) Repeat step 9 for each disk in the virtual machine
11) Restart the virtual machine
12) Run the full backup again and any following differential and incremental backups should now operate correctly
Veritas Corporation has acknowledged that the above-mentioned issue is present in the current version(s) of the product(s) mentioned at the end of this article. Veritas Corporation is committed to product quality and satisfied customers.
This issue is currently under investigation by Veritas Corporation. Pending the outcome of the investigation, this issue may be resolved by way of a patch or hotfix in current or future revisions of the software. However, this particular issue is not currently scheduled for any release. If you feel this issue has a direct business impact for you and your continued use of the product, please contact your Veritas Sales representative or the Veritas Sales group to discuss these concerns. For information on how to contact Veritas Sales, please see https://www.veritas.com
Please be sure to refer back to this document periodically as any changes to the status of the issue will be reflected here.
To confirm which User or Group of users is assigned to a specific Virtual Machine and which role they are assigned, review the permissions screen/tab of the Virtual Machine itself inside the vSphere Client. Additionally, this screen also shows what level the permissions are defined as VMware security does have a concept of security inheritance (propagation).
To review the user and role assignments applied to each datastore (which can also affect the backup and restore processes), inside the vSphere Client, select View --> Inventory --> Datastores and then select the required datastore and click the Permissions tab.
To review the specific permissions applied to each role, in the vSphere Client select View --> Administration --> Roles from the menu. Then right click the role of interest and select "Edit Role..." Note: the built in roles of Read-only and Administrator cannot be edited, but can be cloned to either review the settings or base a custom role around the settings.
In practice, once the correct security is configured, most systems will then perform incremental and differential backups. However some virtual machines may require a one-time reboot due to the running state of the system still denying the ability to enable the Changed Tracking settings. Veritas cannot predict which virtual machines might need a restart.
Workaround:
Should the above solution fail to allow differential and incremental backups to run correctly, then the required ctkEnabled settings can be manually created or modified in the Configuration Parameters screen as a workaround:
1) Make sure the virtual machine is shutdown
2) In the vSphere client, edit the settings of the virtual machine
3) Click on the first Hard Disk and make a note of the SCSI (N:N) reference listed against the Virtual Device Node
4) Repeat step 2 for each disk
5) Click on the Options Tab
6) Under Advanced section select General
7) Click the Configuration Parameters button - this will be grayed out if the virtual machine is running.
8) Use the Add Row button to create a row called ctkEnabled and set to true
9) Add an additional row for the first Hard Disk called scsiN:N.ctkEnabled (where the N:N in the row name matches the information recorded in step 2) and set the value to true
10) Repeat step 9 for each disk in the virtual machine
11) Restart the virtual machine
12) Run the full backup again and any following differential and incremental backups should now operate correctly
Veritas Corporation has acknowledged that the above-mentioned issue is present in the current version(s) of the product(s) mentioned at the end of this article. Veritas Corporation is committed to product quality and satisfied customers.
This issue is currently under investigation by Veritas Corporation. Pending the outcome of the investigation, this issue may be resolved by way of a patch or hotfix in current or future revisions of the software. However, this particular issue is not currently scheduled for any release. If you feel this issue has a direct business impact for you and your continued use of the product, please contact your Veritas Sales representative or the Veritas Sales group to discuss these concerns. For information on how to contact Veritas Sales, please see https://www.veritas.com
Please be sure to refer back to this document periodically as any changes to the status of the issue will be reflected here.
Related Articles
Translated Content
Please note that this document is a translation from English, and may have been machine-translated. It is possible that updates have been made to the original version after this document was translated and published. Veritas does not guarantee the accuracy regarding the completeness of the translation. You may also refer to the English Version of this knowledge base article for up-to-date information.
Get Support
Create Case
Chat
