How one small metric error can make one big business fail

Luke Brown

Luke Brown is the CEO, co-owner and driving force behind Australian agency, Affinity. He has spent more than a decade in integrated comms and media with some of the industry’s top multinationals including Procter & Gamble, Woolworths, Fairfax, Dell and Southcorp. Through this experience, he uncovered a need for a very different type of agency that could benefit all brands. With data analysis at its core, this tech-driven media and ad agency creates personalised connections between people and brands to produce affinity.

If asked to estimate the amount of data used on a daily basis, we’d struggle to come up with an accurate answer. And therein lies the problem. With so much data available, the real challenge lies in realistic ways to manage the sheer volume of information.

Many businesses now use this abundance of data to make critical decisions as a matter of standard practice. But God really is in the detail. A very small mistake in the information used may lead to major faults in the decisions they influence.

One such small error very recently caused an upset for the Australian Bureau of Statistics (ABS), banking industry and the government as a whole, when it came to light that data on first homebuyers in Australia was incorrect.

Prior to the ABS analysis, a number of banks changed the way they classified their mortgage customers. Unaware of the modification in measurement, the ABS, which initially collected and used this information, continued to offer information under its old metric definition.

Based on numbers supplied by the ABS, politicians became charged up over a forthcoming crisis in first-home ownership, spurring hundreds of public servants to review and reset their policies. This led the government to invest millions of dollars into correcting a problem that didn’t exist, all thanks to a single metric definition reporting error.

The lesson to be learned here is two-fold. The first is that big mistakes can arise from very small oversights. The fact that a government body like the ABS is capable of making such a simple error should highlight just how easy it is for a company to be vulnerable.

Secondly, this example highlights why using a single metric to inform business decisions is fraught with danger. Best practice is to use different sources of information across a number of market factors to create indices.

As an organisation, we consistently apply the ‘Ten Measurement’ test (see below) to ensure we’re covering all aspects of the business question, filtering through to what’s really important for any business.

Our 10 measurement tests for any business data

  1. The truth test. Are we really measuring what we set out to measure?
  2. The focus test. Are we only measuring what we set out to measure?
  3. The relevancy test. Is it the right measure to measure the performance we want to track?
  4. The consistency test. Will the data always be collected in the same way whoever measures it?
  5. The access test. Is it easy to locate and capture the data needed to make the measurement?
  6. The clarity test. Is any ambiguity possible in interpreting the results?
  7. The so-what test. Can and will the data be acted upon, i.e. is it actionable?
  8. The timeliness test. Can the data be accessed rapidly and frequently enough for action?
  9. The cost test. Is the measure worth the cost of measurement?
  10. The gaming test. Is the measure likely to encourage undesirable or inappropriate behaviours?

At the end of the day, effective use of data takes more than just reporting the numbers you get. Ensuring your measurement reflects reality, gives you truthful insights and the information you need to take the right action at the right time.

Tags: data-driven

Show Comments

Featured Whitepapers

State of the CMO 2019

CMO’s State of the CMO is an annual industry research initiative aimed at understanding how ...

More whitepapers

Latest Videos

Conversations over a cuppa with CMO: Tyron Hayes

​The current global COVID-19 pandemic is resulting in unprecedented disruption to every aspect of our lives as marketing and brand professionals, from the ways we work, to how our organisations operate, and the way in which we acquire and engage customers. So we’ve drawn on our wonderful CMO50 alumni community to explore different aspects of the crisis facing all of us right now.

More Videos

Why these voice assistants are so popular nowadays? Maybe I should get one too? I am really curious.

Jill Kim

Aussie brands jump on voice-interaction bandwagon following Amazon Alexa's local launch

Read more

Your page is very helpful. Thank you for sharing with us

Eriona Ajvazi

10 brands making a positive difference to a world in crisis

Read more

Extremely insightful and well written. Thanks for the great article!

Nicole Brodie Nahum

Why COVID-19 makes it more important than ever to move at the speed of the consumer

Read more

Blockchain is one of the fastest growing technology in today's digital era. Industries like banking and finance are already using blockch...

Aniket Singh

Can blockchain deliver on its big advertising promises?

Read more

Great article Emma. So many gems in there. Awesome to have you in the team!

One Small Step Collective

Why COVID-19 makes it more important than ever to move at the speed of the consumer

Read more

Blog Posts

The gear change required for business during COVID-19

The current world pandemic, COVID-19, and its tragic effects has created different and challenging situations for nearly every business. Every business sector is affected differently, depending on the nature of what your place in the world, creating the most unique situation most of us have ever and will ever experience during our professional lives.

Katja Forbes

Managing director of Designit, Australia and New Zealand

How can organisations debias their decisions?

​People whose personal details and experiences signal they come from racially diverse backgrounds are less likely than anglo or Caucasian candidates to make it through the first cut in recruitment processes. Even if the organisation says it values diversity.

Dr Karen Morley

Author, commentator

Is your marketing team adapting quickly enough to the COVID-19 crisis?

The impact of coronavirus is far reaching with the true impact on the economy and businesses is unknown. While there are a few categories and brands experiencing growth, for the most part the crisis is wreaking havoc for large and small operators across many sectors including entertainment, tourism, retail, fitness, services and the list goes on.

Teresa Sperti

Founder, Arktic Fox

Sign in