The raw volume of data that businesses accumulate these days is so big that 60% of them budget money specifically for data storage.
It’s not that shocking when you consider that businesses get data from so many sources: customers, social media, vendors, and even from retail checkouts. The trouble comes later when the business needs to manage all the data that pours into their hands.
You don’t want to delete it because it has long-term strategic value. You also don’t want it clogging up the servers if you only access it once a month. What businesses are really up against is figuring out what to do when hot data becomes cold data.
Table of Contents:
The answer to the problem is cold data storage. Before we jump into the nitty-gritty of cold storage, we need to take a look at cold data.
Cold data is more of an idea than a well-defined term. What counts as “cold” for one company might not for another. It comes down to how YOU are using your data. In essence, cold data is data that you only access occasionally or that you no longer use actively.
An easy way to think of cold data is to compare the books on the top or bottom shelf of a bookcase in a professional office.
The books on the bottom shelf have some value but aren’t as easily accessed. That means the person either knows the information, or it’s not relevant to any current professional activities. On the other hand, people put books they use as a frequent reference on the shelves they find easiest to reach.
The reason cold data is hard to pin down is that active use varies from business to business. In one business, data that goes 30 days without getting used might be cold. In another, data might need to sit around for 90 or 180 days before it’s considered cold based on the business cycle.
That takeaway is that every business needs to establish firm ground rules for when data counts as “cold”, so data management remains consistent across their organization.
It’s important to understand that a business may have all of its data stored in the cloud. Hot storage is the top-end, “we need immediate access” kind of storage. That’s the kind of storage used for things like financial transactions and real-time analytics. Essentially data that you will need to access frequently (daily, hourly, or even in real-time).
Cold data storage serves the other data, but it comes with some tradeoffs that we’ll cover in the next few sections.
In certain cases, where security is critical, cold storage means storing data completely offline. For example, you could store data on physical discs or put it on drives with no internet access.
Cold storage providers work from the assumption that, if you access the data, the speed of access won’t be important.
The data is often stored on-premises on servers that use traditional hard drives. The traditional drives cost less than solid state drives and let companies keep costs down. The prices of that frugality are speed and the possibility of drive failure.
Traditional hard drives are much slower than the solid-state drives used for accessing hot data. The internal mechanical parts of traditional drives will wear out eventually. Assuming the company uses some form of basic redundancy, such as RAID, the data remains safe from drive failure.
Cold storage in the cloud is much the same – it’s less expensive than “hot” cloud storage, but also takes longer to access.
Assuming no bizarre situations crop up, the slower access speed shouldn’t prove an issue for most businesses.
Some industries, such as healthcare, have strict rules regarding data access set in place by governing bodies such as the FDA. Cold storage is still available to them, but it needs to be managed differently.
Protocols must be in place to strictly limit who can access the data. You also need something in place, such as an audit trail, to record every and any time the data someone accesses the data. If you are subject to regulations and compliance, you’ll want to make sure your cold storage provider is equipped to meet those needs.
As a general rule, cold data storage is cheaper than hot storage.
As noted above, the hardware is less expensive. That makes maintenance cheaper and helps keep the costs down.
Cold storage providers have the advantage of economies of scale which provides a lower price per GB of data. That means it’s almost always cheaper to store your cold data with them rather than maintain your own cold storage.
There is a catch sometimes. Depending on the company, accessing the data in cold storage more than a certain number of times a month or above a certain amount of data accessed can trigger additional fees. Those fees can quickly eat up the cost savings if you aren’t careful.
Again, this will vary from business to business. If you’re storing data because you think you’ll gain some research advantage from it once or twice, basic cold storage is probably the right answer.
If you expect that you’ll need to access it with any kind of regularity, cold storage is likely not be the best solution.
Let’s assume that you have a good working policy that defines when data is considered cold. Let’s also assume you don’t expect to need serious access to that data. Cold data storage makes your overall data storage and local infrastructure much easier to manage.
It makes your overall data storage easier because you can probably automate the move from hot cloud or local storage to cold cloud storage. If you keep everything in the cloud, the cloud provider can probably automate everything for you based on your policies.
It also means you don’t need to worry about expanding your existing storage capacity all the time. That becomes the storage provider’s problem.
If you keep hot data storage local, your IT infrastructure only needs to keep pace with your existing demands.
Cold data storage isn’t right for every business. If you deal with very sensitive information, the benefits might not outweigh the risks.
Cold storage in the cloud does have limits. Access to the data is slower, courtesy of traditional hard drive technology. You may also lose the cost benefits if you need to start accessing that data on a regular basis.
For most businesses, cold storage is a real answer to the problem of massive data accumulation. It lets the business keep the data for potential future use, without that data clogging up their hot storage.
The business avoids the headache and cost of a constant expansion of local storage. Assuming you develop good data management policies, you can likely automate the data migration from hot to cold storage.