Five lessons from Expedia’s test-and-learn culture

Global president and CEO shares details the culture and operational strategies allowing the online travel to rapidly innovate

For online travel company, Expedia, there is no good or bad idea, you just need to test them.

During a flying visit to Sydney to mark the company’s 10th birthday in Australia, as well as new plans for the Wotif brand acquired last year, president and CEO, Dara Khosrowshahi, revealed how the group is fostering a culture of rapid innovation.

At any one point in time, brand Expedia is working on 50 different versions of the website and providing daily updates to its customers. Its engineering teams can detect small changes and figure out what version a consumer really likes using data insights, then adapt quickly to suit.

“We don’t have to guess, just test,” Khosrowshahi said.

“What we find is one-third of ideas work, one-third are things the consumer doesn’t care about, and one-third fail. The difference between failure and success can be so small. You need to harness audiences that over time will show if you it’s a good or bad feature.”

Here are five lessons CMO took away from Expedia’s test-and-learn approach that every brand can benefit from.

1. Failure is inevitable, so be transparent about it

According to Khosrowshahi, two-thirds of the things Expedia tests don’t work. And that’s quite alright.

“We embrace a culture that accepts failure as we are embracing failure all the time. It’s about speed and innovation,” he said. “Every time we have a failure, we tell everyone about it across the leadership team. We also drive a culture of transparency.”

Helping foster this openness is a monthly ‘product day’, where each team is invited to present on their past 30 days and next 30 days, and encouraged to discuss both wins and failures.

“Failure has to be part of your culture, then you free people up to take shots,” Khosrowshahi said.

2. Iteration is inherent in innovation

Innovation at Expedia is about constantly iterating its products and features. As an example, Khosrowshahi pointed to its Scratchpad offering, which is now available to consumers across on any device. This allows them to store and edit their holiday ideas and planning, as well as share these with people and communities for interactive discussion.

The feature base has been built up over time, Khosrowshahi said, using data-driven customer insight.

“Scratchpad started as a way for a user to look at their history, then it became something available across all devices, then the editing came in, then it allowed you to send a curated list to friends,” he explained. “Change is step by step.”

3. Ideas can come from anybody

Supporting Expedia’s test-and-learn culture is the concept that ideas can come from any part of its business.

“Regardless of how senior you are, if you have an idea, it will get tested and live or die based on that,” Khosrowshahi said. “Previously, a senior person would make that decision. If that happens, staff will spend all their energy on becoming more senior so they can make decisions. But with engineers, you want them coding, not managing people.

“Your idea is as good as my idea, and that’s been a major cultural change with the company. That has evolved over time.”

Ideas are pooled using a general email address, and product owners with responsibility for different parts of Expedia’s sites can access those and choose which ones to test.

“But overall power is dispersed, and there is that transparency too,” Khosrowshahi said.

“Once per month, we can talk about what you have developed in the last 30 days, where the backlog is and plans for the next 30 days. All ideas and presented and discussed and the whole team votes on your pitch.”

4. Don’t be afraid to retest and mix things up

Expedia is constantly A/B testing, but the measurements and objectives driving these tests vary. One example could be testing features for conversion versus satisfaction.

Alongside digital testing, Expedia maintains a consumer lab, which investigates a host of theories and features relating to its products and services, as well as consumer behaviour. One recent test was using electropads on consumers’ faces to detect microgestures as they interacted with Expedia’s digital offering.

“There’s no good or bad idea, need to test them,” Khosrowshahi added.

Wotif Group managing director, David Finch, said the company isn’t afraid to retest three or four times, or to extend the time spent on a test if it’s struggling to get enough data to verify whether a feature change has been successful.

“The challenge is to get a strong enough signal on what we call a ‘beat, meet or lose’,” he said. “Depending on the size of test, we could test for a week, or just days. If the signal’s not clear, we’ll keep trying.”

Tests are also evoked based around different paths and journeys of a customer, Finch said. But he agreed it’s important not to alienate the customer by introducing too many changes at once.

“For example, we won’t update the landing page for three months at a time. It’s about balance – that’s always top of mind,” Finch said. For instance, the group is keeping the Wotif platform stable at the moment after moving the whole site to the Expedia technology platform a few months ago, he said.

5. Testing isn’t just about technology development

While the test-and-learn approach may have started with websites and A/B testing of technology feature sets, it’s now expanded across all parts of the business, Finch said.

“Our marketing teams are looking to test and learn on brand messaging,” he said. “For example, we will run five versions of an ad, then measure site traffic when an ad is on air, and the one creating the biggest bump will be seen as the best creative.

“Rather than one person deciding on what is best, it’s all based on test and learn.”

More on fostering innovation:

Follow CMO on Twitter: @CMOAustralia, take part in the CMO conversation on LinkedIn: CMO ANZ, join us on Facebook: https://www.facebook.com/CMOAustralia, or check us out on Google+: google.com/+CmoAu

Join the CMO newsletter!

Error: Please check your email address.
Show Comments

Blog Posts

Cannes 2017: The Machines Are Here

It’s day 4 in Cannes and among the ever-growing divergent panels, presentations and workshops spanning from one end of the Croisette to the other, there has been a very real emergence of how artificial intelligence (AI) and machine learning needs to fit into the marketing ecosystem of today and tomorrow.

Aden Hepburn

Executive creative director and managing director, VML Australia

Is your content marketing missing the mark?

Does it ever seem like the content you create falls flat on its face or that the leads you’re generating aren’t worth following up?

Dan Ratner

managing director, uberbrand

​ Creating a purpose-driven brand

So you want to be a brand with purpose. But what does it actually mean to build a brand with real meaning?

Paul Chappell

Partner and managing director, Brand + Story

It is interesting. Rebrand is very good. Perhaps they should change the logo to something more modern. For example as it is - https://www...

David Hill

Grace Group undergoes first rebrand in 30 years to unify and contemporise

Read more

Hey Guys, just one small typo that changes a part of the story :“That was a really big step forward for our company because we didn’t hav...

Damian Young

Chobani tastes success with data analytics platform

Read more

This is amazing! Congratulations to Cochlear for continuing to lead innovation in every way!

Chris

How this marketing ops leader is lifting the automation ante at Cochlear

Read more

Interesting case! It seems like universities can also benefit from marketing automation. I've started using getresponse some time ago and...

Aaren

How marketing automation is helping drive CX change at Adelaide University

Read more

Always great to see and read these success stories and the growth of gamification. This story is very warming and can act as an inspirati...

James Doyle

5 tips to boost engagement with gamification from Donut King

Read more

Latest Podcast

More podcasts

Sign in