One of our customers had an incident that perfectly illustrates why businesses need a backup solution for their OneDrive for Business sites.
Here is what happened: Early one morning, an employee’s Surface laptop was infected with some sort of malware which performed 1,754 file deletions in Windows Explorer. All of the affected files were in a shared OneDrive for Business folder that contained important data (e.g., contracts, estimates, project records, and proposals).
In this case, the customer had Veritas NetBackup SaaS Protection in place for their Microsoft 365 backups, so they were able to recover the data quickly and fully. NetBackup SaaS Protection restored the deleted files, and the customer reinstalled the Surface laptop with a clean image.
However, this incident demonstrates just how important it is to have a data protection solution for your OneDrive for Business data.
What would this customer’s scenario be like if they had not been using NetBackup SaaS Protection?
To fully appreciate the native capabilities of Microsoft 365, we have to consider that most organizations subscribing to Microsoft 365 make heavy use of OneDrive for Business. It’s a great tool for collaboration. Enterprises often replace traditional home directories with OneDrive for Business sites. And most workers are probably heavy users of their OneDrive, sharing folders and files as needed, with all sorts of data that is critical to the business.
The following is our assessment of what organizations face (the good, bad, and the ugly) when depending on Microsoft 365 native tools alone.
The OneDrive owner benefited from a feature Microsoft introduced back in August 2018 - sending email notifications of mass file deletion activity. Without this, the issue may not have been detected so easily. Here is the email message our customer received:
In our customer’s case, the employee dismissed the notification message as a phishing attack. It was only later, upon closer inspection, that the user decided it might be legitimate after all.
While it is a great feature, the problem is that Microsoft 365 users often receive several phishing email messages each day that do a very good job at looking like official Microsoft alerts about their Microsoft 365 account.
These phishing attempts have a numbing effect on users — they start to see legitimate Microsoft emails as yet one more phishing attempt.
And that’s a problem because, as the message states,
When files are deleted, they’re stored in your recycle bin and can be restored within 93 days. After 93 days, deleted files are gone forever.”
SharePoint Online and OneDrive for Business data goes to a Microsoft 365 Recycle Bin when it is deleted. The Recycle Bin provides the list of deleted files within the past 90 days. You can select one or more files and either delete or restore them with the click of a button.
Problem solved, right? Unfortunately, no.
We simulated some OneDrive for Business deletion scenarios and did some Recycle Bin testing of our own. Our restore from the Recycle Bin yielded a partial recovery. Approximately 20% of the files were not restored.
The restore error for each file said the file already existed in the location. However, we checked several files for which the Recycle Bin gave this error, and they were NOT in the location.
Upon closer inspection, we noticed that the error details for each file referenced the same file and location. Confusing at first, after more investigation we pieced it together. Microsoft’s code is batching the restore jobs and they aren’t handling item errors elegantly — if one item in the batch fails, it appears that all the remaining items in the batch fail.
If you encounter this problem, one workaround is to click through items in the Recycle Bin, restoring each item one at a time. Painful, especially at scale.
Another option is to roll up your sleeves and write some code. Using the Graph API or SharePoint Client Object Model (CSOM), you could initiate restores of items in the Recycle Bin for a site collection and handle the errors properly yourself to ensure you initiate a separate restore action for each item that needs to be restored.
We also explored using SCOM or PowerShell to gather a copy of the data in the Recycle Bin and write it out to an alternate location, but that doesn’t appear to not be a supported function. Programmatically, just like in the GUI, your only available actions are to either restore a file to its original location – possibly overwriting changes you want to save – or to delete it from the Recycle Bin.
The inability to execute a flawless restore from the Microsoft Recycle Bin is concerning, to say the least, especially if you have a large number of files to recover. Posts in online forums reveal that OneDrive restore scenarios often involve 50,000 or more files deleted accidentally.
With larger numbers of files to restore, you’re more likely to hit the batch restore error scenario and initiating item-level restore jobs in the Recycle Bin GUI is just not practical.
Another place you could recover from is the Windows Recycle Bin on individual workstations. We tested this and it works but it’s almost impossible to understand what you are and aren’t going to be able to recover with this method.
For example, if several users are accessing files from the same OneDrive for Business site collection, and they’re doing so from the desktop, then they will have copies some of the files locally.
If one of the users then performs a mass deletion, the sync engine on each local machine removes the files for the other users. If the other users have these files locally then they’ll go to the Windows Recycle Bin on that machine. If you restore from there, it will put the file back into the local OneDrive folder scope and re-sync.
If one of the users disabled the 'File On-Demand' feature, then they should have a complete list of files for any folders they can access in their Windows Recycle Bin in a mass deletion scenario.
If you’re not thrilled with the Recycle Bin and Microsoft 365's native recovery capabilities you’re not alone. The functionality is too basic to be effective at scale.
Before we look at some recommendations, I thought it was pertinent to call out a major reason for the OneDrive for Business data recovery fire drills you might be having – permissions!
I wish SharePoint Online had a permission level that allowed read, write, and edit BUT NOT delete!
Unfortunately, there is only one permission level that denies delete permission: Read. This is a view-only level of access. There are multiple permission levels, but all except ‘Read’ include the ability to delete!
In my experience, when I share things with others from my OneDrive, I want their contribution to my files, but I don’t want them deleting anything!
Think about this in practice: OneDrive makes it easy to share folders and files, and these shares likely are at the permission levels of Design, Edit, or Contribute – all of which include the power to delete.
Is anyone reviewing and trimming these entitlements as time passes? Not likely in most organizations. That means your OneDrive for Business data is likely exposed to more people within your organization than you realize, many of them having delete control over some- or all – of +your OneDrive files!
Although not achieving a true backup, here are some things you can do in Microsoft 365 to help protect your organization’s information:
None of the above achieve a true, proper backup of your data, and most recovery scenarios are not simple to support.
We offer Veritas NetBackup™ SaaS Protection, which uses a multifaceted approach to protect Microsoft 365 data, including:
In closing, we’ve seen it enough times to know that data deletion – accidental or malicious – is a real threat to businesses. The problem with most Software-as-a-Service (SaaS) applications — whether it be Microsoft 365, Google Workspace, or Slack, as examples — is that they don’t offer true backup and recovery capabilities that work for enterprises.
To learn more about truly protecting your cloud collaboration data, especially Microsoft 365, contact us to schedule a demo.