How natural selection AI is generating huge conversion increases for RateCity

Home loan comparison marketplace shares how an alternative to traditional A/B testing is paying dividends

Traditional A/B testing can be biased in terms of results and limited in terms of designs available for testing. Ultimately, someone has to choose something, and if you end up choosing two ineffective sites for testing, the results will be meaningless.

But not so for RateCity, which engaged an artificial intelligence (AI) solution via Sentient for a recent site testing, which resulted in a 51 per cent uplift in conversions. The solution was so successful, the home loan conversino website is applying the solution across its various verticals moving in the future.

Like many businesses, RateCity could see the value in A/B testing, but didn’t have the time or resources with its small team to undertake the amount of testing required in-house. Ratecity chief technology officer, Pravin Mahajan, said the financial comparison site had lots of website testing already happening, but it wasn’t as effective as they needed it to be.

“We had to write into our framework for what changes we wanted. Then we would look at the results, there was bias involved, and we just ended up in continuous discussion loops within the team, so we were limited by our resources,” he told CMO. “We wanted to remove the subjectivity involved in A/B testing, look at what was statistically significant, and then we could all analyse the detail on a common platform, rather than trying to integrate our own versions."

RateCity needed a solution that could keep speed with its level of innovation without taxing its small team.

“RateCity has always prided itself for its technical leadership in the field of price comparison,” Mahajan said. “So, when we heard Sentient had applied AI to the field of conversion optimisation, we had to give it a try.”

The first test was based on its personal loan funnel. The team came up with 19 variations to try on five different elements of their personal loans landing page. Some were major changes, such as different layouts to the featured product at the top of the page, and some were more granular, such as button colours, button text, and button text size on their key call-to- action button. All in all, there were more than 3000 possible design combinations of the various changes.

As  senior vice-president, strategy and alliances at Sentient Technologies, Jonathan Epstein, explained, Ascend is an AI tool based on evolutionary algorithms, which are modelled on Darwinian natural selection.

"It can test many more changes to a website than A/B testing as the AI efficiently solves what the best combination of variants are out of thousands or millions of potential designs to achieve the best result,” he said. “It is staging generations of designs, looks at the winners, and from that provides a new set of designs. It’s like a farm for websites.

“When you follow the rules of evolution, you accelerate and automate the conversion process.”

After two months of the experiment, Ascend had evolved a design that performed 51 per cent better at converting visitors into leads than the control. And, during the experiment itself, conversions increased by 13.5 per cent, immediately returning the RateCity’s investment.

RateCity CEO, Paul Marshall, said the results saved both time and egos. “There can be bias involved in site testing. We let the data talk for itself, but we certainly weren’t expecting such a huge improvement," he said. 

“When you test one or two things, the results can be underwhelming. Testing a combination of changes, that’s where the power is. The amount of variables we can test is almost unlimited."

As Epstein said, the AI removes the biased slant from testing. “When you have the ability to test more ideas, can be more exploratory, try things you wouldn’t have tried ordinarily. We can test 50 to 60 designs on the way to a final design.”

Mahajan said RateCity’s initial A/B testing in house would increase in single digits only. “We didn’t see how we could use five or six combinations together and come up with something totally different and more effective. We have more sites to get though now and we are also seeing competitors following what we are doing with our sites,” he said.

As a result, RateCity was able to invest its marketing dollars more effectively, as the increased conversion rate meant a much lower acquisition cost per lead. RateCity is now setting up to use Ascend across its largest sales funnels, including credit cards, banking and retirement funds. Experimentation on the personal loan funnel will also continue.

 “The more efficiently we can serve consumers, whether us or our provider partners, the better it is for everyone involved—consumer, provider and RateCity alike,” Mahajan said.

Follow CMO on Twitter: @CMOAustralia, take part in the CMO conversation on LinkedIn: CMO ANZ, join us on Facebook: https://www.facebook.com/CMOAustralia, or check us out on Google+: google.com/+CmoAu

Join the newsletter!

Or

Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.
Show Comments

Latest Videos

Conversations over a cuppa with CMO: Craig Davis

​Leadership resilience, startups scaling up, marketing best practices, customer insights - these are just a few of the topics we manage to explore in the latest episode of Conversations over a Cuppa with CMO featuring Craig Davis.

More Videos

Good day sir / madamWe CLOSED JOINT-STOCK COMPANY AO KAYUM NEFT OIL COMPANY is one of theleading Oil & Gas trading companies in Russi...

BARYBKIN ALEXANDER ALEXANDROVI

3-pronged marketing approach for property disruptor Brickx

Read more

Good day sir / madamWe CLOSED JOINT-STOCK COMPANY AO KAYUM NEFT OIL COMPANY is one of theleading Oil & Gas trading companies in Russi...

BARYBKIN ALEXANDER ALEXANDROVI

Oath to fully acquire Yahoo7 from Seven West Media

Read more

Thank you for sharing your knowledge. Definitely bookmarked for future reading! Check this website https://a2designlab.com/ with lots of ...

Ryota Miyagi

Brene Brown: What it takes to be a brave leader right now

Read more

Well said! It is high time to look into the cultural values and beliefs of the audience before serving with the ads. If it is against the...

Praveen Kumar

The X factor in multicultural media planning and buying - Digital advertising - CMO Australia

Read more

I completely agree with you. High-quality customer service only strengthens loyalty to the company and helps to increase sales and increa...

Natali

Mercer CMO: How B2B brands can achieve customer love

Read more

Blog Posts

Life beyond the cookie: 5 steps to mapping the future of marketing measurement

​There’s no denying there’s been a whirlwind of response to the imminent demise of the third-party cookie from all parts of the industry. But as we’ve collectively come to better understand the implications, it’s clear this change is giving the digital advertising industry the opportunity to re-think digital marketing to support core industry use cases, while balancing consumer privacy.

Natalie Stanbury

Director of research, IAB Australia

Ensuring post-crisis success

The COVID-19 pandemic has exposed brands’ CX shortcomings and a lack of customer understanding. Given ongoing disruption, customer needs, wants and expectations are continually changing, also causing customers to behave in different ways. Just look at hoarding toilet paper, staple and canned food, medicinal and cleaning products.

Riccardo Pasto

senior analyst, Forrester

A few behavioural economics lesson to get your brand on top of the travel list

Understanding the core principles of Behavioural Economics will give players in the travel industry a major competitive advantage when restrictions lift and travellers begin to book again. And there are a few insights in here for the rest of the marketing community, too.

Dan Monheit

Co-founder, Hardhat

Sign in