Even Data-Driven Businesses Should Cultivate Intuition

The desire to make better decisions faster is one of the fundamental drivers of new big data analytics technologies and a general push toward data-driven decision-making. But the relationship between data and intuition -- the old 'gut feeling' -- is a complicated one, says Peter Swabey, senior editor, technology at the Economist Intelligence Unit (EIU), the research and analysis division of The Economist Group.

"They both play a role," Swabey says. "The process of developing data is the process of trying to identify what the true state is. In identifying that, your intuition could be a useful guide."

In an effort to better understand how business decisions are made, predictive analytics firm Applied Predictive Technologies (APT) asked the EIU to conduct a study, resulting in a report released this week: Decisive action: How businesses make decisions and how they could do it better.

In February 2014, EIU surveyed 174 senior managers and executives from around the world and from a range of industries, 49 percent of whom represented organizations with more than US$500 million in annual revenue. EIU also conducted in-depth interviews with practitioners and experts.

The study found that 42 percent of respondents said they collect and analyze data as much as possible before making a decision, while an additional 17 percent said they approach decisions empirically by developing hypotheses and performing tests to prove or disprove them. In other words, 59 percent rely on data to help them make decisions. Of the remainder of respondents, 32 percent said they seek to collaborate on decisions as much as possible, while only 10 percent said they primarily rely on intuition to make decisions.

"Despite the apparent popularity of data-driven decision-making, however, intuition is in fact valued highly," says Jane Bird, author of the EIU report. "Nearly three-quarters of respondents (73 percent) say they trust their own intuition when it comes to decision-making. Even among the data-driven decision-makers, over two-thirds (68 percent) agree with that statement."

And, Bird notes, 68 percent of them would be trusted by their peers and superiors to make a decision not supported by data.

"However much data you take in, and whatever the interview process, when you get to the end there has to be an element of gut feel too," says Alison Robb, group director for people, customer, communications and commercial at Nationwide Building Society, one of the expert practitioners interviewed for the EIU report. "It's partly chemistry, experience and knowing what you do and don't like."

Intuition Can Help Identify Problems With Data Analysis

Perhaps nothing illustrates the inextricable link between data-driven decisions and human intuition better than how people would respond if the available data contradicted gut feeling. Fully 57 percent of survey respondents said that if confronted with that situation, the first thing they would do is reanalyze the data; 30 percent said they would collect more data and only 10 percent said they would take the course of action suggested by the data.

As an example, Rupert Naylor, vice president of APT, says that just last week, an APT client ran a test of the effectiveness of putting inserts into newspapers: Would they drive more sales?

"They ran the analysis and the results came back completely flat," Naylor says. "There was no difference between the performance of the tests and the control."

Surprised by the results, which on the face of it suggested that inserts don't drive additional sales at all, the client asked APT to check whether anything had gone wrong with the analytics software. But APT found the analysis had been run properly. Dumbfounded, the client spoke to the media agency responsible for the test, only to discover that the agency had failed to run it.

"That was a great victory for data," Naylor says. "The data was saying that nothing had happened."

"It's also a vindication of intuition," Swabey adds. "It's the fact that the result was counterintuitive that indicated something was wrong."

Even as organizations move to become more data-driven in their decision-making, Bird says the survey findings suggest that intuition should also be valued and cultivated.

"Evidently both intuition and analysis contribute to effective decision-making, in business as in life," she says. "Rather than a weakness that must be avoided, intuition should instead be seen as a skill that is appropriate in the right circumstances."

In fact, she suggests that to improve the nature of decision-making within organizations, those organizations must acknowledge that decision-making itself is a standalone skill that can be improved just like other skills. Furthermore, she says, organizations must hold executives accountable for the quality of their decisions.

In the report, 19 percent of respondents said decision-makers in their organizations are not held accountable at all, while 59 percent acknowledged that poor decision-makers will still be allowed to progress within their organization. And it may be that this state of affairs persists due to a lack of transparency in decision-making. Two-thirds (64 percent) of respondents said information about who makes decisions and why is restricted to sufficiently senior employees; only 26 percent said that information is freely available to everyone.

"This implies that in a significant minority of organizations, senior managers take decisions behind closed doors for opaque reasons and, should those decisions turn out to be the wrong ones, are not held accountable," Bird says. "This is a disturbing state of affairs."

Gerry Grimstone, chairman of insurer Standard Life, says it is essential to cultivate accountability while at the same time ensuring that executives don't exist in a climate of fear that prevents them from raising serious issues until it is too late.

"I'm a great believer in accountability, and I'm happy that my board members have got some liability," he says. "They take better decisions if they think that there are reputational consequences if they get it wrong."

Join the CMO newsletter!

Error: Please check your email address.
Show Comments

Supporting Association

Blog Posts

Disruption Down Under – What’s Amazon’s real competitive advantage?

Savvy shoppers wait in anticipation, while Australian retailers are gearing up for the onslaught. Amazon’s arrival is imminent.

Change across the board: Why boards need to digitally evolve

Traditionally the non-executive board of a company acts in an advisory capacity - attending monthly board meetings to offer overarching advice and guidance typically focusing on:

Jodie Sangster

CEO, ADMA

The most desirable customers you’ve overlooked

“What will really move the needle?” This is a question that keeps leaders awake at night. And at the intersection of some of their top priorities – finding pockets of growth, redefining the customer experience, and making an emotional impact – lies a latent market: Their diverse customers.

Really inspiring !

Goldenboy Media

Jaywing sets sights on Australian growth with digital and data-driven agency model

Read more

Being aware of regulations or guidlines is just the start. As our CEO Emma Lo Russo stated exactly two weeks ago at an event we supported...

Alan Smith

​Are the Wild West days of influencer collaboration over?

Read more

Rebranding is always nice solution to get better organisation. Businessman may apply certain special services (for example, https://www.l...

David Hill

CMO interview: Spearheading the global rebranding of OFX

Read more

Thank you so much for sharing this article.Top Digital Marketing company in Bangalore

Way To DM

Predictions: 17 digital marketing trends for 2017

Read more

Thanks for the great article Jodie, agree many boards and senior execs are operating in outdated modes, just as we need some reverse soci...

sharyn

Change across the board: Why boards need to digitally evolve - Data-driven marketing - CMO Australia

Read more

Latest Podcast

More podcasts

Sign in