4 barriers stand between you and Big Data insight

Over the past three years, the buzz around big data has continued to grow. While some dismiss it as so much hype, businesses around the world are taking notice: In its recently released 5th annual Digital IQ Survey, consulting firm PwC found 62 per cent of respondents believe big data can give them a competitive advantage.

But believing in the power of big data is one thing; leveraging big data for actionable insights is another. PwC also found that 58 per cent of respondents agree that moving from data to insight is a major challenge.

"The amount of information and data that we're collecting now is truly enormous in terms of the volume that is outside the four walls of the organization," says Anand Rao, principal at PwC. "Organisations don't have the right people, they don't have the right structure in place and they're still struggling with some of the tools and techniques."

PwC surveyed 1,108 respondents from 12 countries and across a variety of industries. The respondents were evenly divided between IT and business leaders, and more than 75 per cent worked in organisations with revenues of more than US$1 billion. PwC found that organisations struggle with four major big data barriers:

• They're blind to the importance of visualisation
• They're investing more in gathering data than analysing it
• They're facing a talent gap
• They're struggling with insufficient systems to rapidly process information.

Businesses blind to the importance of visualisation

When it comes to actually deriving insight from the trove of data at most organisations' disposal, visualisation is fundamental. Visualisation helps put data into context and bring business cases to life. In many cases, advanced visualisation capabilities allow organisations to glean insights that would be impossible otherwise.

For instance, due to archaic records and inaccurate information, most utilities have no idea where all of their underground assets are located, resulting in all-too-common service interruptions for residents when a power line is accidently cut or a water line bursts.

To avoid these problems, the City of Las Vegas took advantage of smart data to develop a living model of its utilities network. VTN Consulting helped the city aggregate data from various sources into a single real-time 3D model created with Autodesk technology. The model includes both above and below ground utilities, and is being used to visualise the location and performance of critical assets located under the city.

And yet, only 26 per cent of global survey respondents are using data visualisation. But the picture is very different when you focus in on top performing companies - those respondents that reported revenue growth of greater than five percent and whose companies are in the top quartile for revenue, profitability and innovation. Those organisations lead the pack in terms of plans to invest more in data visualisation in 2013.

"The kind of visualisation that most people are moving toward is having dashboards of information where you can zoom in or zoom out - easily digestible to the business user," says Rao. "It's more hindsight analysis, basically backward looking. Most organisations are getting that. But now that we understand what happened or did not happen based on actions we took, we want to look into the future. That requires visualisation that is much more dynamic."

Investing more in gathering data than analysing it

Companies are investing significant amounts in gathering data, Rao and Halter say, but perhaps not enough to integrate, merge and analyse it: 32 per cent of organisations have invested more than US$1 million in gathering, storing and retrieving internal data, but only 26 per cent have invested more than $1 million in analysis of internal data.

Rao and Halter say respondents from the financial services, insurance and healthcare industries appear to be investing the most in data integration, and one-third of top performers likewise are investing more than $1 million in integrating third-party data.

Organisations may be hoarding data without analysing it because IT and the business are locked into old ways of working with data.

"The traditional model of how business and IT work together traditionally no longer works in this field," says Oliver Halter, principal at PwC. "Business and IT always have had difficulties talking to each other. Traditionally, business creates requirements and IT executes. In the world of exploring data, that doesn't work so well anymore. The knee-jerk reaction of IT is, 'We're going to collect data and manage it, and you guys figure out what to do with that.'"

But the sophisticated analysis required to glean meaning from big data is often beyond business users.

"Those things are complicated and your typical business user doesn't know about it," Halter says. "I think what we're looking for is a new organisational approach, which means new talent and new ways of exploring that data."

They're facing a talent gap

And that leads to the third big data barrier: the talent gap. As Rao and Halter note, it's no secret companies often lack talent in the skills necessary to interpret big data. Only 44 per cent of PwC's survey respondents said they have a sufficient pipeline of talent to undertake deep analysis of data, though top performers were more likely to feel they have a sufficient talent pipeline.

But Rao and Halter say that companies often overlook their existing talent: individuals in marketing analysis, actuarial groups and pricing/product development. These individuals can serve as a great starting point for talent to translate data into insight.

"Organisations that have been successful early have created new organisational models," Halter says. "They've created centres of excellence where business and IT come together. I have worked with clients where we literally created entirely new structures on the side of the IT organisation. The teams have to be much more nimble, much more agile."

"The business person needs to understand more about data analytics, visualisation, all of that," adds Rao. "You need someone who understands the business, can translate the business from a problem into a solution. They have to understand enough about analytics to know that this type of problem requires that type of solution or analytics technique. If you can't find a single person, a team approach could work."

Struggling with insufficient systems to rapidly process information

The fourth big data barrier is existing systems. Rao and Halter note that big data demands increased computing power to rapidly gather, store and analyse massive volumes of data.

But many organisations doubt their ability to do so with their current systems. Forty-one per cent of respondents in the Americas, 33 per cent of respondents in Europe and 49 per cent of respondents in Asia-Pacific said their systems can't process large volumes of data from different sources. Even top performers mostly aligned with the pack with regard to confidence in their processing power.

But Rao and Halter say they believe that organisations often struggle with this issue because they're looking at the entirety of the data available to them rather than focusing on a particular problem.

"When companies don't have a hypothesis or problem in hand that they want to solve, that's when they a big data overload," Rao says.

"There's all this information out there and they don't know what to do with it. If you approach it from a perspective of a particular problem like, 'I'm trying to grow a market with this specific segment,' it becomes much clearer. Most organisations are just overwhelmed with all the information available rather than focusing just on the problem that has to be solved."

That's also the best way to get a rapid return on investment to a big data problem, Rao says.

"The best way to get your ROI is to home in on a couple of key questions," he says. "Get a group of people solely focused on that."

Halter adds that there is one thing that the companies most successful in leveraging big data have in common.

"The companies that achieve success, there's one thing that unites them," he says. "All of them out there ahead of the curve have a very senior executive that for whatever reason saw the potential early and just rammed it through the organisation."

"I think big data is becoming more and more an umbrella message," he adds. "It's not just big data, it's data everywhere. It's how do you use data big and small to get ahead of the curve. There's no way out of it. It's not just large files and Hadoop; it's really about changing your culture to a more analytical culture."