Dark Data Offers Both Opportunity and Risk

In some ways, the cloud is a double-edged sword. The ease with which we can spin up servers lets businesses to align their infrastructure portfolio to their business needs in close to real time. On the other hand, without proper management, it's also easy to deploy resources that are not used to their fullest potential.

The problem of unused cloud servers gets a fair bit of attention, but the cloud is about data storage as much as compute resources, and the problem of so-called dark data is on the rise.

Dark data is one of those terms that industry analysts are still fighting over, so its meaning has not yet been settled. It's been defined by some as potentially useful data that could be harvested from business processes but is currently not leveraged. But it's a related definition we're considering here: data that has been harvested and that sits in cloud storage racking up fees while businesses do nothing with it.

Dark data comes about for many different reasons: data was collected for a purpose that was never acted on, compliance demands, ineffective data retention and deletion policies, forgotten projects, and so on. If companies are to maximize the cost-effectiveness of their cloud infrastructure, they must implement policies for dealing with data before it becomes dark, and for managing it once it is dark.