Even Data-Driven Businesses Should Cultivate Intuition

The desire to make better decisions faster is one of the fundamental drivers of new big data analytics technologies and a general push toward data-driven decision-making. But the relationship between data and intuition -- the old 'gut feeling' -- is a complicated one, says Peter Swabey, senior editor, technology at the Economist Intelligence Unit (EIU), the research and analysis division of The Economist Group.

"They both play a role," Swabey says. "The process of developing data is the process of trying to identify what the true state is. In identifying that, your intuition could be a useful guide."

In an effort to better understand how business decisions are made, predictive analytics firm Applied Predictive Technologies (APT) asked the EIU to conduct a study, resulting in a report released this week: Decisive action: How businesses make decisions and how they could do it better.

In February 2014, EIU surveyed 174 senior managers and executives from around the world and from a range of industries, 49 percent of whom represented organizations with more than US$500 million in annual revenue. EIU also conducted in-depth interviews with practitioners and experts.

The study found that 42 percent of respondents said they collect and analyze data as much as possible before making a decision, while an additional 17 percent said they approach decisions empirically by developing hypotheses and performing tests to prove or disprove them. In other words, 59 percent rely on data to help them make decisions. Of the remainder of respondents, 32 percent said they seek to collaborate on decisions as much as possible, while only 10 percent said they primarily rely on intuition to make decisions.

"Despite the apparent popularity of data-driven decision-making, however, intuition is in fact valued highly," says Jane Bird, author of the EIU report. "Nearly three-quarters of respondents (73 percent) say they trust their own intuition when it comes to decision-making. Even among the data-driven decision-makers, over two-thirds (68 percent) agree with that statement."

And, Bird notes, 68 percent of them would be trusted by their peers and superiors to make a decision not supported by data.

"However much data you take in, and whatever the interview process, when you get to the end there has to be an element of gut feel too," says Alison Robb, group director for people, customer, communications and commercial at Nationwide Building Society, one of the expert practitioners interviewed for the EIU report. "It's partly chemistry, experience and knowing what you do and don't like."

Intuition Can Help Identify Problems With Data Analysis

Perhaps nothing illustrates the inextricable link between data-driven decisions and human intuition better than how people would respond if the available data contradicted gut feeling. Fully 57 percent of survey respondents said that if confronted with that situation, the first thing they would do is reanalyze the data; 30 percent said they would collect more data and only 10 percent said they would take the course of action suggested by the data.

As an example, Rupert Naylor, vice president of APT, says that just last week, an APT client ran a test of the effectiveness of putting inserts into newspapers: Would they drive more sales?

"They ran the analysis and the results came back completely flat," Naylor says. "There was no difference between the performance of the tests and the control."

Surprised by the results, which on the face of it suggested that inserts don't drive additional sales at all, the client asked APT to check whether anything had gone wrong with the analytics software. But APT found the analysis had been run properly. Dumbfounded, the client spoke to the media agency responsible for the test, only to discover that the agency had failed to run it.

"That was a great victory for data," Naylor says. "The data was saying that nothing had happened."

"It's also a vindication of intuition," Swabey adds. "It's the fact that the result was counterintuitive that indicated something was wrong."

Even as organizations move to become more data-driven in their decision-making, Bird says the survey findings suggest that intuition should also be valued and cultivated.

"Evidently both intuition and analysis contribute to effective decision-making, in business as in life," she says. "Rather than a weakness that must be avoided, intuition should instead be seen as a skill that is appropriate in the right circumstances."

In fact, she suggests that to improve the nature of decision-making within organizations, those organizations must acknowledge that decision-making itself is a standalone skill that can be improved just like other skills. Furthermore, she says, organizations must hold executives accountable for the quality of their decisions.

In the report, 19 percent of respondents said decision-makers in their organizations are not held accountable at all, while 59 percent acknowledged that poor decision-makers will still be allowed to progress within their organization. And it may be that this state of affairs persists due to a lack of transparency in decision-making. Two-thirds (64 percent) of respondents said information about who makes decisions and why is restricted to sufficiently senior employees; only 26 percent said that information is freely available to everyone.

"This implies that in a significant minority of organizations, senior managers take decisions behind closed doors for opaque reasons and, should those decisions turn out to be the wrong ones, are not held accountable," Bird says. "This is a disturbing state of affairs."

Gerry Grimstone, chairman of insurer Standard Life, says it is essential to cultivate accountability while at the same time ensuring that executives don't exist in a climate of fear that prevents them from raising serious issues until it is too late.

"I'm a great believer in accountability, and I'm happy that my board members have got some liability," he says. "They take better decisions if they think that there are reputational consequences if they get it wrong."

Join the CMO newsletter!

Error: Please check your email address.
Show Comments

Supporting Association

Blog Posts

Getting your business ready for the Entrepreneurial Consumer

We all know the digital revolution has completely transformed the way consumers are interacting with brands, and that a lot of businesses are finding it hard to catch up. One way to closing this brand gap is to understand consumer behaviour and build a brand experience that meets these new needs.

Pip Stocks

CEO and founder, BrandHook

Brand management starts with management

As the world continues to grow and evolve, it’s more important than ever to build a strong brand that articulates your message clearly and consistently, stands out against the noise, and develops relevance with the people that matter. This makes managing your brand a key component to gaining cut-through and ultimately business success.

Dan Ratner

managing director, Uberbrand

Disrupting marketing as we know it

Call it digital disruption or the fourth industrial revolution, our rapidly evolving environment is affecting consumer perceptions, purchase behaviours and the way they consume information and products.

Jean-Luc Ambrosi

Author, marketer

Very interesting article which touches on the importance of a feedback loop fuelled by customer and market insights. Ideally this scenari...

Andrew Reid

Building customer insights in the data and digital age

Read more

Very very good piece- very novel and innovative and very possibly- effective - way to look at one's communication headlines!

Patrick Dsouza

Should your disclaimer become your headline? - Brand science - CMO Australia

Read more

Excellent post Rob, Mobile app users are growing day by day. Everyday lots of apps are launched in the market but not every app retains t...

Marcus Miller

Why app engagement must be personalised - Mobile strategy - CMO Australia

Read more

very informative blog. I really like the information given in this blog.http://gng.com.au/

Gajanand Choudhary

The evolving role of the CMO - The CMO view - CMO Australia

Read more

It is true That’s the new read following up Deloitte Digital's Digital disruption - Short Fuse, blowup analysis series, that appearance t...

miller645645@mail.ru

Digital disruption about to impact health, education sectors

Read more

Latest Podcast

More podcasts

Sign in