Even Data-Driven Businesses Should Cultivate Intuition

The desire to make better decisions faster is one of the fundamental drivers of new big data analytics technologies and a general push toward data-driven decision-making. But the relationship between data and intuition -- the old 'gut feeling' -- is a complicated one, says Peter Swabey, senior editor, technology at the Economist Intelligence Unit (EIU), the research and analysis division of The Economist Group.

"They both play a role," Swabey says. "The process of developing data is the process of trying to identify what the true state is. In identifying that, your intuition could be a useful guide."

In an effort to better understand how business decisions are made, predictive analytics firm Applied Predictive Technologies (APT) asked the EIU to conduct a study, resulting in a report released this week: Decisive action: How businesses make decisions and how they could do it better.

In February 2014, EIU surveyed 174 senior managers and executives from around the world and from a range of industries, 49 percent of whom represented organizations with more than US$500 million in annual revenue. EIU also conducted in-depth interviews with practitioners and experts.

The study found that 42 percent of respondents said they collect and analyze data as much as possible before making a decision, while an additional 17 percent said they approach decisions empirically by developing hypotheses and performing tests to prove or disprove them. In other words, 59 percent rely on data to help them make decisions. Of the remainder of respondents, 32 percent said they seek to collaborate on decisions as much as possible, while only 10 percent said they primarily rely on intuition to make decisions.

"Despite the apparent popularity of data-driven decision-making, however, intuition is in fact valued highly," says Jane Bird, author of the EIU report. "Nearly three-quarters of respondents (73 percent) say they trust their own intuition when it comes to decision-making. Even among the data-driven decision-makers, over two-thirds (68 percent) agree with that statement."

And, Bird notes, 68 percent of them would be trusted by their peers and superiors to make a decision not supported by data.

"However much data you take in, and whatever the interview process, when you get to the end there has to be an element of gut feel too," says Alison Robb, group director for people, customer, communications and commercial at Nationwide Building Society, one of the expert practitioners interviewed for the EIU report. "It's partly chemistry, experience and knowing what you do and don't like."

Intuition Can Help Identify Problems With Data Analysis

Perhaps nothing illustrates the inextricable link between data-driven decisions and human intuition better than how people would respond if the available data contradicted gut feeling. Fully 57 percent of survey respondents said that if confronted with that situation, the first thing they would do is reanalyze the data; 30 percent said they would collect more data and only 10 percent said they would take the course of action suggested by the data.

As an example, Rupert Naylor, vice president of APT, says that just last week, an APT client ran a test of the effectiveness of putting inserts into newspapers: Would they drive more sales?

"They ran the analysis and the results came back completely flat," Naylor says. "There was no difference between the performance of the tests and the control."

Surprised by the results, which on the face of it suggested that inserts don't drive additional sales at all, the client asked APT to check whether anything had gone wrong with the analytics software. But APT found the analysis had been run properly. Dumbfounded, the client spoke to the media agency responsible for the test, only to discover that the agency had failed to run it.

"That was a great victory for data," Naylor says. "The data was saying that nothing had happened."

"It's also a vindication of intuition," Swabey adds. "It's the fact that the result was counterintuitive that indicated something was wrong."

Even as organizations move to become more data-driven in their decision-making, Bird says the survey findings suggest that intuition should also be valued and cultivated.

"Evidently both intuition and analysis contribute to effective decision-making, in business as in life," she says. "Rather than a weakness that must be avoided, intuition should instead be seen as a skill that is appropriate in the right circumstances."

In fact, she suggests that to improve the nature of decision-making within organizations, those organizations must acknowledge that decision-making itself is a standalone skill that can be improved just like other skills. Furthermore, she says, organizations must hold executives accountable for the quality of their decisions.

In the report, 19 percent of respondents said decision-makers in their organizations are not held accountable at all, while 59 percent acknowledged that poor decision-makers will still be allowed to progress within their organization. And it may be that this state of affairs persists due to a lack of transparency in decision-making. Two-thirds (64 percent) of respondents said information about who makes decisions and why is restricted to sufficiently senior employees; only 26 percent said that information is freely available to everyone.

"This implies that in a significant minority of organizations, senior managers take decisions behind closed doors for opaque reasons and, should those decisions turn out to be the wrong ones, are not held accountable," Bird says. "This is a disturbing state of affairs."

Gerry Grimstone, chairman of insurer Standard Life, says it is essential to cultivate accountability while at the same time ensuring that executives don't exist in a climate of fear that prevents them from raising serious issues until it is too late.

"I'm a great believer in accountability, and I'm happy that my board members have got some liability," he says. "They take better decisions if they think that there are reputational consequences if they get it wrong."

Join the newsletter!

Or

Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.
Show Comments

Latest Videos

More Videos

More Brand Posts

What are Chris Riddell's qualifications to talk about technology? What are the awards that Chris Riddell has won? I cannot seem to find ...

Tareq

Digital disruption isn’t disruption anymore: Why it’s time to refocus your business

Read more

Enterprisetalk

Mark

CMO's top 10 martech stories for the week - 9 June

Read more

Great e-commerce article!

Vadim Frost

CMO’s State of CX Leadership 2022 report finds the CX striving to align to business outcomes

Read more

Are you searching something related to Lottery and Lottery App then Agnito Technologies can be a help for you Agnito comes out as a true ...

jackson13

The Lottery Office CEO details journey into next-gen cross-channel campaign orchestration

Read more

Thorough testing and quality assurance are required for a bug-free Lottery Platform. I'm looking forward to dependability.

Ella Hall

The Lottery Office CEO details journey into next-gen cross-channel campaign orchestration

Read more

Blog Posts

Marketing prowess versus the enigma of the metaverse

Flash back to the classic film, Willy Wonka and the Chocolate Factory. Television-obsessed Mike insists on becoming the first person to be ‘sent by Wonkavision’, dematerialising on one end, pixel by pixel, and materialising in another space. His cinematic dreams are realised thanks to rash decisions as he is shrunken down to fit the digital universe, followed by a trip to the taffy puller to return to normal size.

Liz Miller

VP, Constellation Research

Why Excellent Leadership Begins with Vertical Growth

Why is it there is no shortage of leadership development materials, yet outstanding leadership is so rare? Despite having access to so many leadership principles, tools, systems and processes, why is it so hard to develop and improve as a leader?

Michael Bunting

Author, leadership expert

More than money talks in sports sponsorship

As a nation united by sport, brands are beginning to learn money alone won’t talk without aligned values and action. If recent events with major leagues and their players have shown us anything, it’s the next generation of athletes are standing by what they believe in – and they won’t let their values be superseded by money.

Simone Waugh

Managing Director, Publicis Queensland

Sign in