Five lessons from Expedia’s test-and-learn culture

Global president and CEO shares details the culture and operational strategies allowing the online travel to rapidly innovate

For online travel company, Expedia, there is no good or bad idea, you just need to test them.

During a flying visit to Sydney to mark the company’s 10th birthday in Australia, as well as new plans for the Wotif brand acquired last year, president and CEO, Dara Khosrowshahi, revealed how the group is fostering a culture of rapid innovation.

At any one point in time, brand Expedia is working on 50 different versions of the website and providing daily updates to its customers. Its engineering teams can detect small changes and figure out what version a consumer really likes using data insights, then adapt quickly to suit.

“We don’t have to guess, just test,” Khosrowshahi said.

“What we find is one-third of ideas work, one-third are things the consumer doesn’t care about, and one-third fail. The difference between failure and success can be so small. You need to harness audiences that over time will show if you it’s a good or bad feature.”

Here are five lessons CMO took away from Expedia’s test-and-learn approach that every brand can benefit from.

1. Failure is inevitable, so be transparent about it

According to Khosrowshahi, two-thirds of the things Expedia tests don’t work. And that’s quite alright.

“We embrace a culture that accepts failure as we are embracing failure all the time. It’s about speed and innovation,” he said. “Every time we have a failure, we tell everyone about it across the leadership team. We also drive a culture of transparency.”

Helping foster this openness is a monthly ‘product day’, where each team is invited to present on their past 30 days and next 30 days, and encouraged to discuss both wins and failures.

“Failure has to be part of your culture, then you free people up to take shots,” Khosrowshahi said.

2. Iteration is inherent in innovation

Innovation at Expedia is about constantly iterating its products and features. As an example, Khosrowshahi pointed to its Scratchpad offering, which is now available to consumers across on any device. This allows them to store and edit their holiday ideas and planning, as well as share these with people and communities for interactive discussion.

The feature base has been built up over time, Khosrowshahi said, using data-driven customer insight.

“Scratchpad started as a way for a user to look at their history, then it became something available across all devices, then the editing came in, then it allowed you to send a curated list to friends,” he explained. “Change is step by step.”

3. Ideas can come from anybody

Supporting Expedia’s test-and-learn culture is the concept that ideas can come from any part of its business.

“Regardless of how senior you are, if you have an idea, it will get tested and live or die based on that,” Khosrowshahi said. “Previously, a senior person would make that decision. If that happens, staff will spend all their energy on becoming more senior so they can make decisions. But with engineers, you want them coding, not managing people.

“Your idea is as good as my idea, and that’s been a major cultural change with the company. That has evolved over time.”

Ideas are pooled using a general email address, and product owners with responsibility for different parts of Expedia’s sites can access those and choose which ones to test.

“But overall power is dispersed, and there is that transparency too,” Khosrowshahi said.

“Once per month, we can talk about what you have developed in the last 30 days, where the backlog is and plans for the next 30 days. All ideas and presented and discussed and the whole team votes on your pitch.”

4. Don’t be afraid to retest and mix things up

Expedia is constantly A/B testing, but the measurements and objectives driving these tests vary. One example could be testing features for conversion versus satisfaction.

Alongside digital testing, Expedia maintains a consumer lab, which investigates a host of theories and features relating to its products and services, as well as consumer behaviour. One recent test was using electropads on consumers’ faces to detect microgestures as they interacted with Expedia’s digital offering.

“There’s no good or bad idea, need to test them,” Khosrowshahi added.

Wotif Group managing director, David Finch, said the company isn’t afraid to retest three or four times, or to extend the time spent on a test if it’s struggling to get enough data to verify whether a feature change has been successful.

“The challenge is to get a strong enough signal on what we call a ‘beat, meet or lose’,” he said. “Depending on the size of test, we could test for a week, or just days. If the signal’s not clear, we’ll keep trying.”

Tests are also evoked based around different paths and journeys of a customer, Finch said. But he agreed it’s important not to alienate the customer by introducing too many changes at once.

“For example, we won’t update the landing page for three months at a time. It’s about balance – that’s always top of mind,” Finch said. For instance, the group is keeping the Wotif platform stable at the moment after moving the whole site to the Expedia technology platform a few months ago, he said.

5. Testing isn’t just about technology development

While the test-and-learn approach may have started with websites and A/B testing of technology feature sets, it’s now expanded across all parts of the business, Finch said.

“Our marketing teams are looking to test and learn on brand messaging,” he said. “For example, we will run five versions of an ad, then measure site traffic when an ad is on air, and the one creating the biggest bump will be seen as the best creative.

“Rather than one person deciding on what is best, it’s all based on test and learn.”

More on fostering innovation:

Follow CMO on Twitter: @CMOAustralia, take part in the CMO conversation on LinkedIn: CMO ANZ, join us on Facebook: https://www.facebook.com/CMOAustralia, or check us out on Google+: google.com/+CmoAu

Join the CMO newsletter!

Error: Please check your email address.
Show Comments

Supporting Association

Blog Posts

Behaviour change, by design

​We’re living in an age of unprecedented change. We experience with Oculus Rift, invest with Acorns, consume video through Hyper, tune into Pandora and navigate with Waze.

Glen Jeffreys

Head of UX, Deepend Group

Chat bots: How to use them commercially right now

I’m sure that many of you out there have heard a lot about chat bots (aka messaging bots) recently, and the fact that they are here to stay is pretty evident by now.

Deniz Nalbantoglu

Managing director, Webling-Interactive

Top tips to uncovering consumer insights for business innovation

An in-depth understanding of consumers sits at the heart of what we all need to do, but we know it’s not always easy to uncover insights that will unlock a true innovation opportunity.

Matt Whale

Managing director, How To Impact

Thanks, Nadia. It was so interesting blog for me because it helps me able to understand well on the basics of business and marketing conc...

MichaelBGreen

3 ways marketers can raise their executive influence

Read more

Having been to the AO on Wednesday for the first time in a number of years I was very impressed with the new branding. The event has evol...

Naomi

Tennis Australia unwraps new brand identity for Australian Open

Read more

This was amazing and so inspiring. Thanks for sharing.

Vijay Kumar

Design thinking: Leading with experience

Read more

Great read, thanks for posting. MR should be seen as the holy grail for marketers and brands, as it offers an unprecedented capability to...

Barney

Interview: The business case for mixed reality in marketing

Read more

what a load of shit, and what a major stuff up... the new brand device is sterile, boring and just plain bad. Would be better suited to a...

James Yoi

Tennis Australia unwraps new brand identity for Australian Open

Read more

Latest Podcast

More podcasts

Sign in