How Telstra is applying machine learning to marketing mix modelling

Telstra's analytics leader shares how a new modelling solution is transforming the way the marketing function understands effectiveness


Understanding the impact and optimal combination of marketing and media tactics for success is an increasingly complicated task for marketing functions. It’s even more difficult if you want to interpret the data in a quick enough timeframe to optimise, pivot or swap out marketing programs while they’re still in-market.

But this is exactly what Telstra is becoming able to do thanks to a new marketing mix modelling approach based on machine learning, rolled out in June.

Telstra director of research, insights and analytics, Liz Moore, told CMO the ASX-listed telco built an econometric approach to marketing mix modelling about five years ago based on the need to better understand the ROI of marketing at a strategic level. While it satisfied one organisational need – namely, giving the CFO a view of marketing ROI – it was an expensive exercise, taking 4-5 months to gather more than 300 data sources, plus months to build the modelling.

“Eight to nine months later, we could tell how effective your marketing mix was 12 months ago,” she said. “It wasn’t providing marketers with real-time flexibility of understanding what is going on in the marketplace. The challenge we wanted to solve, was that for every dollar we are spending, how effective is it? And how do we do that fast enough to take decisions off the insights being made around the CMO function around how to go to market?

“We knew continuing to take the traditional econometric approach wasn’t right. We were spending more than returns we were getting, given it was being delivered so late after the fact.”

Telstra went to market for a new approach that was fast, cost efficient and incredibly insightful, Moore said. In addition, any solution needed to sit in the ecosystem of tools being used measure marketing and not be done in a vacuum chamber. These include Aprimo’s Marketing Studio operations management tool, a bottom-up attribution modelling solution fuelling Telstra’s programmatic trading desk, brand and ad tracking, and also uplift over control measurements across every one-to-one marketing campaign undertaken by the team.

“That’s important because if we look at a campaign or channel has performed well or poorly, we know it can be for three reasons: There’s an investment issue; there’s a creative issue; or there’s an offer or competitive issue,” Moore commented.  

The solution: Machine learning

Telstra partnered with Accenture to tap new machine learning technologies that could drive improved speed to insight. Known internally as ‘triple M’,  the marketing mix modelling solution is refresh monthly, instead of every six months, and available on the desktop of users across the business within a month of closing off a month, once data sets are all in. The tool is tapping more than 4000 data variables, most of which are automated, bringing speed and agility that’s unprecedented, Moore said.

“It’s put in the hands of media buyers, marketers, CMOs and finance and to the board a close to real-time assessment of how marketing is going so we can make strategic decisions about whether we are investing in the right places, with that strategic overlay as well,” she said.  

As part of the development process, Moore and her team worked with marketers, media buyers and agency partners to understand constraints and criteria, and devise modelling simulations and recommendations that both made sense and would be useful. It was also vital the tool provide a through-the-line, cross-segment, integrated view of marketing, Moore said.

Many of the data assets fuelling the model were already in the hands of the Telstra analytics team, including feedback from media buyers at a granular level. The problem was integration was largely manual, with some sent in spreadsheets and sent at different times. A key step in development of ‘triple M’ was working with providers to get data provided in a systematic, automated fashion, Moore said.

The modelling also includes Telstra data variables around sales, footfall, activations, deactivations, along with competitive landscape information.

“A key lesson is that many assume data feeds are going to be won and done in projects like this. But data is a bit like a garden, you have to constantly tender it,” Moore said.  

Accenture Analytics managing director, Marek Rucinski, said it was vital to gain an understanding of the interaction of channels in terms of conversions and ROI.

“It’s a broader problem for marketers – how digital interacts with traditional channels,” he said. “Many are struggling with that intellectually. We now have a detailed picture of these cross-channel interactions, which can then be presented visually to the team.”

Rucinski said the tool incorporates three core data packets: Performance data; marketing spend data such as campaign details, native to advertising, reach; and controlled data sets, such as competitor performance, pricing movement. Driving the model is machine learning and open source platforms that allow data and models to be processed more quickly.

“It now takes us 11 days to run more than 900 models simultaneously. You can’t underestimate the processing power needed,” he explained. “It’s self-learning, and every cycle changes based on patterns. That introduces its own challenges as you have to ensure the team can explain what the model is generating.

“Other continual change is the business doesn’t stand still. This needs to be made to evolve with the business, otherwise you have the tail wagging the dog.”

As a case in point, Moore said Telstra’s launch of brand 3.0 last year half-way through development of the new marketing mix modelling tool changed the structure of how the business goes to market and meant a rebuild of all the models previously created.

“As the business changes and evolves, we have to continue changing the way we measure and think about marketing modelling,” she said. “Machine learning allows us to make those changes in a much more agile way.  

Rucinski also described the presentation layer of Telstra’s new marketing mix modelling platform as “radically different”.

“We’ve moved from static reports and heavy desktop tools to something that is light on and fact-based, with different screens for different users such as the executives, who get a broader view; or a campaign manager, who can drill down into specific campaign and how it’s performed,” he said.  

“The key to success is creating a sense of belief about the results. If it doesn’t relate to a marketer’s function or reality, they will ignore them. Telstra and the team worked to create a reference set that creates a common language. That’s a major achievement.”

Cultural impact

While the maths behind the modelling is complex, Moore agreed it’s the cultural component that often proves the hardest part of rolling out such a solution.

“Without culture, the stickiness just isn’t there and you never get to your destination,” she said. “What was important was our CMO was 120 per cent behind this, wanting us to move to making decisions that are data and fact-based. It’s that support from the top that drove take-up throughout the organisation.

“It’s embedded in the way we look and assess campaigns now. People are now saying ‘have we have run it through triple M, what are the insights?’. Plus, it’s been built into the operating rhythm and a tool that supports the CMO.”

As part of the rollout, Moore’s team worked with Deakin University’s professor of marketing, Chris Dubelaar, to review maps.

“We were pushing the envelope so we wanted an academic who could test the thinking and provide input and work with us,” Moore said. “It was useful for the modelling and our team to have that because what we’re doing is so bleeding edge.”  

Already, ‘triple M’  is bringing a degree of experimentation to Telstra’s above-the-line investment, Moore continued.

“We look at a channel giving us a higher ROI than thought, and this allows us to say let’s test that theory, and put more investment in one geography and see how it performs against another geography,” she said. “It’s a measurement tool for assessing experiments.”  

But the tool itself isn’t the whole solution and both Moore and Rucinski emphasised how important it is to consider the wider marketing context.

“We also need to include qualitative and quantitative research data to see what it is telling us,” Moore said. “That’s the art beside the science. This is a bedrock and important part of our measurement ecosystem, but we need the other pieces to work well together too.”

To ensure its hundreds of marketers don’t “run down data rabbit holes”, Moore said the group has limited access to those who need to use this on a day-to-day basis. That includes the marketing media buyers and CMO leadership team. To help, the analytics team has conducted monthly deep dives, bringing in insights to support particular campaign planning and trading cycles.

There are also specific individuals in the operations team of each CMO function and the media agency utilising the modelling tool today.

Moore said the next big step forward will be moving the tool from being monthly to weekly.

“That allows us to pick up underlying performance view and assess on a weekly basis to allow for inflight tuning,” she said, adding the next phase is due in the New Year.

Follow CMO on Twitter: @CMOAustralia, take part in the CMO conversation on LinkedIn: CMO ANZ, join us on Facebook: https://www.facebook.com/CMOAustralia, or check us out on Google+:google.com/+CmoAu

Join the newsletter!

Or

Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.
Show Comments

Latest Videos

Launch Marketing Council Episode 3: Launching in the technology sector

Our multi-part video series, Ready to Launch, is focused on unlocking the secrets of launching brands, products and services by exploring real-life examples from Australia’s marketing elite. The series is being produced as part of the Launch Marketing Council initiative by CMO in conjunction with independent agency, Five by Five Global.

More Videos

NetSuite started out as a cloud-based provider of Enterprise Resource Planning software or as NetSuite solution provider, which companies...

talalyousaf

NetSuite to acquire Bronto's digital marketing platform for US$200m

Read more

Thanks for sharing this post, its really good information I get through this blog.CDPO Online Exam Training

Infosectrain01

3 ways Booking.com is improving its B2B marketing game

Read more

Time is of the essence, especially for customer service teams. With chatbots, you can interact and assist customers at a larger scale, al...

Jai

Triple-digit customer database growth, personalised engagement become reality for Stone & Wood

Read more

Hey Emilie - great read, and I particularly liked the section on the pressure of having brand purpose/Gen Z spending habits. It's great t...

Chris Thomas

Have customers really changed? - Marketing edge - CMO Australia

Read more

Extremely informative. One should definitely go through the blog in order to know different aspects of the Retail Business and retail Tec...

Sheetal Kamble

SAP retail chief: Why more retailers need to harness data differently

Read more

Blog Posts

How the CMO can get the board on the customer’s side

For some CMOs, it’s easy to feel alone in the undying quest to better serve the customer. At times, it feels like the marketing department and the boards are speaking a different language, with one side trying to serve the customer, and the other side more focused on the shareholders and financials.

Jeff Cooper

CMO and board, Business Excellence Australia

The Secret Ingredients of a CX-Led Company Culture

When I talk to organisations around the world about their customer experience strategy, it is often the CMOs and their marketing teams who take the lead. They’re keen to improve the ways they attract and engage customers, and they want to understand the technologies that can help them make their customer experience truly outstanding.

Steven van Belleghem

Author, CX expert

The Future Of Social Is Joyful, Pass It On

2019 was a horror year for social media. But in 2020 something different emerged that has shifted the tone, format and intent of the medium. A new social vibe born out of the pandemic and fuelled by the emergence of a platform tailor made for the next generation of consumers.

Dan Young

Managing director, Pulse

Sign in