How natural selection AI is generating huge conversion increases for RateCity

Home loan comparison marketplace shares how an alternative to traditional A/B testing is paying dividends

Traditional A/B testing can be biased in terms of results and limited in terms of designs available for testing. Ultimately, someone has to choose something, and if you end up choosing two ineffective sites for testing, the results will be meaningless.

But not so for RateCity, which engaged an artificial intelligence (AI) solution via Sentient for a recent site testing, which resulted in a 51 per cent uplift in conversions. The solution was so successful, the home loan conversino website is applying the solution across its various verticals moving in the future.

Like many businesses, RateCity could see the value in A/B testing, but didn’t have the time or resources with its small team to undertake the amount of testing required in-house. Ratecity chief technology officer, Pravin Mahajan, said the financial comparison site had lots of website testing already happening, but it wasn’t as effective as they needed it to be.

“We had to write into our framework for what changes we wanted. Then we would look at the results, there was bias involved, and we just ended up in continuous discussion loops within the team, so we were limited by our resources,” he told CMO. “We wanted to remove the subjectivity involved in A/B testing, look at what was statistically significant, and then we could all analyse the detail on a common platform, rather than trying to integrate our own versions."

RateCity needed a solution that could keep speed with its level of innovation without taxing its small team.

“RateCity has always prided itself for its technical leadership in the field of price comparison,” Mahajan said. “So, when we heard Sentient had applied AI to the field of conversion optimisation, we had to give it a try.”

The first test was based on its personal loan funnel. The team came up with 19 variations to try on five different elements of their personal loans landing page. Some were major changes, such as different layouts to the featured product at the top of the page, and some were more granular, such as button colours, button text, and button text size on their key call-to- action button. All in all, there were more than 3000 possible design combinations of the various changes.

As  senior vice-president, strategy and alliances at Sentient Technologies, Jonathan Epstein, explained, Ascend is an AI tool based on evolutionary algorithms, which are modelled on Darwinian natural selection.

"It can test many more changes to a website than A/B testing as the AI efficiently solves what the best combination of variants are out of thousands or millions of potential designs to achieve the best result,” he said. “It is staging generations of designs, looks at the winners, and from that provides a new set of designs. It’s like a farm for websites.

“When you follow the rules of evolution, you accelerate and automate the conversion process.”

After two months of the experiment, Ascend had evolved a design that performed 51 per cent better at converting visitors into leads than the control. And, during the experiment itself, conversions increased by 13.5 per cent, immediately returning the RateCity’s investment.

RateCity CEO, Paul Marshall, said the results saved both time and egos. “There can be bias involved in site testing. We let the data talk for itself, but we certainly weren’t expecting such a huge improvement," he said. 

“When you test one or two things, the results can be underwhelming. Testing a combination of changes, that’s where the power is. The amount of variables we can test is almost unlimited."

As Epstein said, the AI removes the biased slant from testing. “When you have the ability to test more ideas, can be more exploratory, try things you wouldn’t have tried ordinarily. We can test 50 to 60 designs on the way to a final design.”

Mahajan said RateCity’s initial A/B testing in house would increase in single digits only. “We didn’t see how we could use five or six combinations together and come up with something totally different and more effective. We have more sites to get though now and we are also seeing competitors following what we are doing with our sites,” he said.

As a result, RateCity was able to invest its marketing dollars more effectively, as the increased conversion rate meant a much lower acquisition cost per lead. RateCity is now setting up to use Ascend across its largest sales funnels, including credit cards, banking and retirement funds. Experimentation on the personal loan funnel will also continue.

 “The more efficiently we can serve consumers, whether us or our provider partners, the better it is for everyone involved—consumer, provider and RateCity alike,” Mahajan said.

Follow CMO on Twitter: @CMOAustralia, take part in the CMO conversation on LinkedIn: CMO ANZ, join us on Facebook: https://www.facebook.com/CMOAustralia, or check us out on Google+: google.com/+CmoAu

Join the newsletter!

Or

Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.
Show Comments
cmo-xs-promo

Latest Videos

More Videos

Thanks for your feedback, Rabi. While we introduced the ROO concept using a marketing example, I also believe that it is pertinent to man...

Iggy Pintado

Introducing Return on Outcome (ROO) - Brand science - CMO Australia

Read more

Thanks for your insight, Philip. Return On Outcome (ROO) requires balanced thinking with the focus on outcomes as opposed to returns.

Iggy Pintado

Introducing Return on Outcome (ROO) - Brand science - CMO Australia

Read more

Beautiful article.

Hodlbaba

15 brands jumping into NFTs

Read more

"Blue" is really gorgeous and perfectly imitates a human customer support operator. Personally, I won't order a chatbot development for m...

Nate Ginsburg

Why the newest member of BT’s contact centre is a chatbot

Read more

As today’s market changes rapidly, the tools we use change, and it is important to adapt to those changes to continue to succeed in busin...

Anna Duda

Report: 10 digital commerce trends here to stay

Read more

Blog Posts

How the pandemic revealed the antidote to marketing’s image problem

What does marketing truly ‘own’ in most organisations? Brand and campaigns, definitely. Customer experience? That remains contested ground.

Murray Howe

Founder, The Markitects

Still pursuing a 360-degree view of the customer?

On the Internet, nobody knows you’re a dog.” It may have been true in 1993 when this caption to a Peter Steiner cartoon appeared in the New Yorker. But after 30 years online, it’s no longer the case.

Agility in 2022

Only the agile will survive and thrive in this environment and that’s why in 2022, agility will need to be a whole-business priority.

Sam McConnell

Melbourne bureau chief, Alpha Digital

Sign in