Skip to content

May 2, 2019 | Matt Pacheco

Three Big Data Challenges Organizations Must Address Now

It’s Not the Data, It’s What You Do with It

Big data is a fact of life. In a 2018 report on analytics, research firm McKinsey & Co put it this way:

“Data is now like air. It’s all around us. It has become common knowledge that the world churns out an enormous and expanding amount of data each day—billions of gigabytes, in fact.”

~ Analytics Comes of Age, McKinsey Analytics, January 2018

For business leaders looking to create a competitive advantage, the increasing availability of data is a welcome trend. However, their ability to take advantage of data will be highly dependent on their capacity to handle what IDC calls the “data deluge.”

You might also like: Five Key Trends Impacting Digital Transformation in 2019

1. Digital transformation will require greater focus on data management

In the early 2000s, as data started flowing in from new sources, big data discussions centered around data analytics. For example, websites could now track customer behaviors such as which pages buyers visited before making a decision, how long they stayed on those pages, and whether they downloaded any additional assets. By analyzing actual behaviors, we could get a clearer picture of customer habits and preferences and use that knowledge to improve our market reach.

Fast forward to today, and we’re light years ahead of creating go-to-market strategies based on website traffic patterns. Since its inception in 1999, the Internet of Things (IoT) has grown from a collection of RFID-connected devices to include the more than 27 billion devices expected to be connected to the internet by the end of this year. (Statista)

These devices are a prime driver of the billions of gigabytes being churned out daily, but thanks to a corresponding growth in computing power, more data is definitely better. As we continue to collect data through distributed, connected devices and systems, advanced algorithms analyze the new information, identifying patterns and offering new insights that we’d never have been able to glean from a spreadsheet analysis. With the power of the IoT and machine learning/AI, big data offers organizations a path by which they can achieve their digital transformation vision.

However, digital transformation will require a sophisticated approach to data management. One particular challenge will be the distributed nature of data, especially as edge data centers, which move data and applications closer to end users, become more prevalent. Quality of data across data centers can become a problem if management policies and practices differ. Going forward, organizations will need to make data governance a large part of their data management strategy.

2. Data security and privacy must remain a priority

Organizations intent on using big data to drive their business forward may find themselves quickly mired in a public relations battle over data privacy. While it’s easy to watch the trials and tribulations of companies like Facebook and think, “Glad that’s not my organization,” executives need to understand that these news stories are increasing general distrust in how data is being used and protected.

The insurance industry, one in which the use of big data holds great promise to both the consumer and the carrier, is a case in point. For example, “lifestyle” data can give insurers a more accurate picture of an applicant’s risk profile while allowing those committed to a healthy lifestyle to lower their rates. That’s the theory at least. In practice, it’s easy to paint a picture that shows how access to this type of data might be abused.

Also read: Turn Compliance into a Competitive Advantage in 2019

Consumers appear quick to believe those stories. In fact, in a new study of 1000 American adults, 72% of respondents did not think carriers should be allowed to use lifestyle data to assess insurance risk. When asked which posed a bigger threat to their personal privacy, tech companies accessing their private data to run targeted ads or insurance companies accessing their private data to assess risk, 55% said they posed equal risk. Another 10% thought insurance companies posed a greater risk.

Organizations looking to leverage big data to advance their objectives need to establish clear guidelines for how the data will be used (and not used). They also need to increase transparency of their data security and privacy practices or risk decreasing consumer trust in their organizations.

3. Acceptable latency will need to be redefined

If you’ve been using technology for any length of time, you know that fast is never fast enough. There’s a reason we upgrade computers every two to three years. We simply don’t have time to wait for our older processors to handle today’s workloads.

Latency extends to network connections as well. As more and more data is transmitted between the data center and an increasingly remote workforce, network congestion can become an issue. This latency is exacerbated by the distances the data has to travel. To make matters worse, technologies like 5G networks are decreasing what we find to be acceptable wait times.

Organizations will need to find ways to address these issues, e.g., deploying more edge data centers (which move the data closer to the end users) and leveraging 5G technologies.

How do you manage your data?

Do you have your own data center, or do you use a data center provider? Read our Strategic Guide to Cloud Computing to learn about the different cloud platforms and how to choose the right mix for your business.

The Strategic Guide to Cloud Computing - read now

Subscribe to the TierPoint blog

We’ll send you a link to new blog posts whenever we publish, usually once a week.