Even Data-Driven Businesses Should Cultivate Intuition

The desire to make better decisions faster is one of the fundamental drivers of new big data analytics technologies and a general push toward data-driven decision-making. But the relationship between data and intuition -- the old 'gut feeling' -- is a complicated one, says Peter Swabey, senior editor, technology at the Economist Intelligence Unit (EIU), the research and analysis division of The Economist Group.

"They both play a role," Swabey says. "The process of developing data is the process of trying to identify what the true state is. In identifying that, your intuition could be a useful guide."

In an effort to better understand how business decisions are made, predictive analytics firm Applied Predictive Technologies (APT) asked the EIU to conduct a study, resulting in a report released this week: Decisive action: How businesses make decisions and how they could do it better.

In February 2014, EIU surveyed 174 senior managers and executives from around the world and from a range of industries, 49 percent of whom represented organizations with more than US$500 million in annual revenue. EIU also conducted in-depth interviews with practitioners and experts.

The study found that 42 percent of respondents said they collect and analyze data as much as possible before making a decision, while an additional 17 percent said they approach decisions empirically by developing hypotheses and performing tests to prove or disprove them. In other words, 59 percent rely on data to help them make decisions. Of the remainder of respondents, 32 percent said they seek to collaborate on decisions as much as possible, while only 10 percent said they primarily rely on intuition to make decisions.

"Despite the apparent popularity of data-driven decision-making, however, intuition is in fact valued highly," says Jane Bird, author of the EIU report. "Nearly three-quarters of respondents (73 percent) say they trust their own intuition when it comes to decision-making. Even among the data-driven decision-makers, over two-thirds (68 percent) agree with that statement."

And, Bird notes, 68 percent of them would be trusted by their peers and superiors to make a decision not supported by data.

"However much data you take in, and whatever the interview process, when you get to the end there has to be an element of gut feel too," says Alison Robb, group director for people, customer, communications and commercial at Nationwide Building Society, one of the expert practitioners interviewed for the EIU report. "It's partly chemistry, experience and knowing what you do and don't like."

Intuition Can Help Identify Problems With Data Analysis

Perhaps nothing illustrates the inextricable link between data-driven decisions and human intuition better than how people would respond if the available data contradicted gut feeling. Fully 57 percent of survey respondents said that if confronted with that situation, the first thing they would do is reanalyze the data; 30 percent said they would collect more data and only 10 percent said they would take the course of action suggested by the data.

As an example, Rupert Naylor, vice president of APT, says that just last week, an APT client ran a test of the effectiveness of putting inserts into newspapers: Would they drive more sales?

"They ran the analysis and the results came back completely flat," Naylor says. "There was no difference between the performance of the tests and the control."

Surprised by the results, which on the face of it suggested that inserts don't drive additional sales at all, the client asked APT to check whether anything had gone wrong with the analytics software. But APT found the analysis had been run properly. Dumbfounded, the client spoke to the media agency responsible for the test, only to discover that the agency had failed to run it.

"That was a great victory for data," Naylor says. "The data was saying that nothing had happened."

"It's also a vindication of intuition," Swabey adds. "It's the fact that the result was counterintuitive that indicated something was wrong."

Even as organizations move to become more data-driven in their decision-making, Bird says the survey findings suggest that intuition should also be valued and cultivated.

"Evidently both intuition and analysis contribute to effective decision-making, in business as in life," she says. "Rather than a weakness that must be avoided, intuition should instead be seen as a skill that is appropriate in the right circumstances."

In fact, she suggests that to improve the nature of decision-making within organizations, those organizations must acknowledge that decision-making itself is a standalone skill that can be improved just like other skills. Furthermore, she says, organizations must hold executives accountable for the quality of their decisions.

In the report, 19 percent of respondents said decision-makers in their organizations are not held accountable at all, while 59 percent acknowledged that poor decision-makers will still be allowed to progress within their organization. And it may be that this state of affairs persists due to a lack of transparency in decision-making. Two-thirds (64 percent) of respondents said information about who makes decisions and why is restricted to sufficiently senior employees; only 26 percent said that information is freely available to everyone.

"This implies that in a significant minority of organizations, senior managers take decisions behind closed doors for opaque reasons and, should those decisions turn out to be the wrong ones, are not held accountable," Bird says. "This is a disturbing state of affairs."

Gerry Grimstone, chairman of insurer Standard Life, says it is essential to cultivate accountability while at the same time ensuring that executives don't exist in a climate of fear that prevents them from raising serious issues until it is too late.

"I'm a great believer in accountability, and I'm happy that my board members have got some liability," he says. "They take better decisions if they think that there are reputational consequences if they get it wrong."

Join the CMO newsletter!

Error: Please check your email address.
Show Comments

Supporting Association

Blog Posts

Why app engagement must be personalised

Research from Nielsen late last year reported Australian smartphone users over the age of 18 spend 33 hours per month in apps, and a mere four hours per month in browsers. But what does it take to actually maintain an app customers will engage with?

Rob Marston

Head of Airwave, A/NZ

Customer experience investments more vital than ever

The global commodity slump has hit Australia in the last few months. Companies that obsess over these developments might be tempted to cut spending on customer experience (CX) programs. Here's why that's a a terrible idea.

Harley Manning and Thomas McCann

Research leaders, Forrester

Managing brands in a digital world

With digital integration at the core of customer management, many marketers have been questioning whether the principles and approaches to branding are fundamentally different in a digitally led environment.

Jean-Luc Ambrosi

Author, marketer

Now you make creative Facebook ads

Mike Smith

Why AAMI turned to Facebook mobile and segmentation to drive brand favourability

Read more

Rob - great article. Here at Pure Oxygen Labs we could not agree more. When considering retail mobile apps deep linking is woefully unde...

Scott

Why app engagement must be personalised - Mobile strategy - CMO Australia

Read more

Project Leader?? Kim Portrate is one of the most ineffective leaders I have ever had the displeasure of meeting. She single-handedly cost...

Anonymous

Helloworld scraps CMO role

Read more

What tripe. This article conveniently makes no mention of her lies and bullying tactics and how she had placed everyone off-side with her...

Anonymous

Helloworld scraps CMO role

Read more

You mentioned cashback sites giving "immediate earnings" for transactions through their site. Cashback sites can take a couple of months...

RG

Are points-based customer loyalty programs on the way out?

Read more

Latest Podcast

More podcasts

Sign in