How natural selection AI is generating huge conversion increases for RateCity

Home loan comparison marketplace shares how an alternative to traditional A/B testing is paying dividends

Traditional A/B testing can be biased in terms of results and limited in terms of designs available for testing. Ultimately, someone has to choose something, and if you end up choosing two ineffective sites for testing, the results will be meaningless.

But not so for RateCity, which engaged an artificial intelligence (AI) solution via Sentient for a recent site testing, which resulted in a 51 per cent uplift in conversions. The solution was so successful, the home loan conversino website is applying the solution across its various verticals moving in the future.

Like many businesses, RateCity could see the value in A/B testing, but didn’t have the time or resources with its small team to undertake the amount of testing required in-house. Ratecity chief technology officer, Pravin Mahajan, said the financial comparison site had lots of website testing already happening, but it wasn’t as effective as they needed it to be.

“We had to write into our framework for what changes we wanted. Then we would look at the results, there was bias involved, and we just ended up in continuous discussion loops within the team, so we were limited by our resources,” he told CMO. “We wanted to remove the subjectivity involved in A/B testing, look at what was statistically significant, and then we could all analyse the detail on a common platform, rather than trying to integrate our own versions."

RateCity needed a solution that could keep speed with its level of innovation without taxing its small team.

“RateCity has always prided itself for its technical leadership in the field of price comparison,” Mahajan said. “So, when we heard Sentient had applied AI to the field of conversion optimisation, we had to give it a try.”

The first test was based on its personal loan funnel. The team came up with 19 variations to try on five different elements of their personal loans landing page. Some were major changes, such as different layouts to the featured product at the top of the page, and some were more granular, such as button colours, button text, and button text size on their key call-to- action button. All in all, there were more than 3000 possible design combinations of the various changes.

As  senior vice-president, strategy and alliances at Sentient Technologies, Jonathan Epstein, explained, Ascend is an AI tool based on evolutionary algorithms, which are modelled on Darwinian natural selection.

"It can test many more changes to a website than A/B testing as the AI efficiently solves what the best combination of variants are out of thousands or millions of potential designs to achieve the best result,” he said. “It is staging generations of designs, looks at the winners, and from that provides a new set of designs. It’s like a farm for websites.

“When you follow the rules of evolution, you accelerate and automate the conversion process.”

After two months of the experiment, Ascend had evolved a design that performed 51 per cent better at converting visitors into leads than the control. And, during the experiment itself, conversions increased by 13.5 per cent, immediately returning the RateCity’s investment.

RateCity CEO, Paul Marshall, said the results saved both time and egos. “There can be bias involved in site testing. We let the data talk for itself, but we certainly weren’t expecting such a huge improvement," he said. 

“When you test one or two things, the results can be underwhelming. Testing a combination of changes, that’s where the power is. The amount of variables we can test is almost unlimited."

As Epstein said, the AI removes the biased slant from testing. “When you have the ability to test more ideas, can be more exploratory, try things you wouldn’t have tried ordinarily. We can test 50 to 60 designs on the way to a final design.”

Mahajan said RateCity’s initial A/B testing in house would increase in single digits only. “We didn’t see how we could use five or six combinations together and come up with something totally different and more effective. We have more sites to get though now and we are also seeing competitors following what we are doing with our sites,” he said.

As a result, RateCity was able to invest its marketing dollars more effectively, as the increased conversion rate meant a much lower acquisition cost per lead. RateCity is now setting up to use Ascend across its largest sales funnels, including credit cards, banking and retirement funds. Experimentation on the personal loan funnel will also continue.

 “The more efficiently we can serve consumers, whether us or our provider partners, the better it is for everyone involved—consumer, provider and RateCity alike,” Mahajan said.

Follow CMO on Twitter: @CMOAustralia, take part in the CMO conversation on LinkedIn: CMO ANZ, join us on Facebook: https://www.facebook.com/CMOAustralia, or check us out on Google+: google.com/+CmoAu

Join the newsletter!

Or

Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.
Show Comments

Blog Posts

Cannes Part 1: Why brands must put human interactions at the heart of their business

As a Media Juror at this year’s Cannes Lions, I was fortunate enough to attend the world’s most influential festival of creativity and listen to thought-leading marketers from around the globe.

Nickie Scriven

CEO, Zenith

4 creative skills that will be useful forever

In recent times, the clarion call from futurists, economists, marketers, educators and leaders the world over is one of slight panic, “The world is changing and you’re not ready for it!” And of course, they make a very good point.

Kieran Flanagan and Dan Gregory

Speakers, trainers, co-authors

Why defining brand strategy is vital to capitalising on quick wins

Big brands were once protected from small brands by high barriers to entry. Big brands had the resources to employ big agencies, to crack big ideas and to invest in big campaigns. They had the luxury of time to debate strategies and work on long-term innovation pipelines. Retailers used to partner with big brands.

Troy McKinnna

Co-founder, Agents of Spring, Calm & Stormy

An interesting update considering that today is the easiest way it has ever been to measure contribution to the business as well as the h...

Frederic

State of the CMO 2019: Tenure shortens, pressure is on as marketers strive to demonstrate impact

Read more

I thought this was what Salesforce Audience Studio (formerly Salesforce DMP) was supposed to do. How are a CDP and a DMP different? I'm c...

Tony Ahn

Salesforce announces customer data platform

Read more

Well written Vanessa!! Agreed with your view that human experience is marketing's next frontier. Those businesses who are focused on the ...

Clyde Griffith

Forget customer experience, human experience is marketing's next frontier

Read more

Great tips for tops skills need to develop and stay competitive

Nick

The top skills needed to stay competitive in a rapidly changing workforce

Read more

The popularity of loyalty programs is diminishing, though I'd say it is because customers are savvy enough to recognise when a loyalty pr...

Heather

It’s time for marketers to rethink their approach to ‘loyalty’

Read more

Latest Podcast

More podcasts

Sign in