Five lessons from Expedia’s test-and-learn culture

Global president and CEO shares details the culture and operational strategies allowing the online travel to rapidly innovate

For online travel company, Expedia, there is no good or bad idea, you just need to test them.

During a flying visit to Sydney to mark the company’s 10th birthday in Australia, as well as new plans for the Wotif brand acquired last year, president and CEO, Dara Khosrowshahi, revealed how the group is fostering a culture of rapid innovation.

At any one point in time, brand Expedia is working on 50 different versions of the website and providing daily updates to its customers. Its engineering teams can detect small changes and figure out what version a consumer really likes using data insights, then adapt quickly to suit.

“We don’t have to guess, just test,” Khosrowshahi said.

“What we find is one-third of ideas work, one-third are things the consumer doesn’t care about, and one-third fail. The difference between failure and success can be so small. You need to harness audiences that over time will show if you it’s a good or bad feature.”

Here are five lessons CMO took away from Expedia’s test-and-learn approach that every brand can benefit from.

1. Failure is inevitable, so be transparent about it

According to Khosrowshahi, two-thirds of the things Expedia tests don’t work. And that’s quite alright.

“We embrace a culture that accepts failure as we are embracing failure all the time. It’s about speed and innovation,” he said. “Every time we have a failure, we tell everyone about it across the leadership team. We also drive a culture of transparency.”

Helping foster this openness is a monthly ‘product day’, where each team is invited to present on their past 30 days and next 30 days, and encouraged to discuss both wins and failures.

“Failure has to be part of your culture, then you free people up to take shots,” Khosrowshahi said.

2. Iteration is inherent in innovation

Innovation at Expedia is about constantly iterating its products and features. As an example, Khosrowshahi pointed to its Scratchpad offering, which is now available to consumers across on any device. This allows them to store and edit their holiday ideas and planning, as well as share these with people and communities for interactive discussion.

The feature base has been built up over time, Khosrowshahi said, using data-driven customer insight.

“Scratchpad started as a way for a user to look at their history, then it became something available across all devices, then the editing came in, then it allowed you to send a curated list to friends,” he explained. “Change is step by step.”

3. Ideas can come from anybody

Supporting Expedia’s test-and-learn culture is the concept that ideas can come from any part of its business.

“Regardless of how senior you are, if you have an idea, it will get tested and live or die based on that,” Khosrowshahi said. “Previously, a senior person would make that decision. If that happens, staff will spend all their energy on becoming more senior so they can make decisions. But with engineers, you want them coding, not managing people.

“Your idea is as good as my idea, and that’s been a major cultural change with the company. That has evolved over time.”

Ideas are pooled using a general email address, and product owners with responsibility for different parts of Expedia’s sites can access those and choose which ones to test.

“But overall power is dispersed, and there is that transparency too,” Khosrowshahi said.

“Once per month, we can talk about what you have developed in the last 30 days, where the backlog is and plans for the next 30 days. All ideas and presented and discussed and the whole team votes on your pitch.”

4. Don’t be afraid to retest and mix things up

Expedia is constantly A/B testing, but the measurements and objectives driving these tests vary. One example could be testing features for conversion versus satisfaction.

Alongside digital testing, Expedia maintains a consumer lab, which investigates a host of theories and features relating to its products and services, as well as consumer behaviour. One recent test was using electropads on consumers’ faces to detect microgestures as they interacted with Expedia’s digital offering.

“There’s no good or bad idea, need to test them,” Khosrowshahi added.

Wotif Group managing director, David Finch, said the company isn’t afraid to retest three or four times, or to extend the time spent on a test if it’s struggling to get enough data to verify whether a feature change has been successful.

“The challenge is to get a strong enough signal on what we call a ‘beat, meet or lose’,” he said. “Depending on the size of test, we could test for a week, or just days. If the signal’s not clear, we’ll keep trying.”

Tests are also evoked based around different paths and journeys of a customer, Finch said. But he agreed it’s important not to alienate the customer by introducing too many changes at once.

“For example, we won’t update the landing page for three months at a time. It’s about balance – that’s always top of mind,” Finch said. For instance, the group is keeping the Wotif platform stable at the moment after moving the whole site to the Expedia technology platform a few months ago, he said.

5. Testing isn’t just about technology development

While the test-and-learn approach may have started with websites and A/B testing of technology feature sets, it’s now expanded across all parts of the business, Finch said.

“Our marketing teams are looking to test and learn on brand messaging,” he said. “For example, we will run five versions of an ad, then measure site traffic when an ad is on air, and the one creating the biggest bump will be seen as the best creative.

“Rather than one person deciding on what is best, it’s all based on test and learn.”

More on fostering innovation:

Follow CMO on Twitter: @CMOAustralia, take part in the CMO conversation on LinkedIn: CMO ANZ, join us on Facebook: https://www.facebook.com/CMOAustralia, or check us out on Google+: google.com/+CmoAu

Join the CMO newsletter!

Error: Please check your email address.
Show Comments

Supporting Association

Blog Posts

Disruption Down Under – What’s Amazon’s real competitive advantage?

Savvy shoppers wait in anticipation, while Australian retailers are gearing up for the onslaught. Amazon’s arrival is imminent.

Change across the board: Why boards need to digitally evolve

Traditionally the non-executive board of a company acts in an advisory capacity - attending monthly board meetings to offer overarching advice and guidance typically focusing on:

Jodie Sangster

CEO, ADMA

The most desirable customers you’ve overlooked

“What will really move the needle?” This is a question that keeps leaders awake at night. And at the intersection of some of their top priorities – finding pockets of growth, redefining the customer experience, and making an emotional impact – lies a latent market: Their diverse customers.

Really inspiring !

Goldenboy Media

Jaywing sets sights on Australian growth with digital and data-driven agency model

Read more

Being aware of regulations or guidlines is just the start. As our CEO Emma Lo Russo stated exactly two weeks ago at an event we supported...

Alan Smith

​Are the Wild West days of influencer collaboration over?

Read more

Rebranding is always nice solution to get better organisation. Businessman may apply certain special services (for example, https://www.l...

David Hill

CMO interview: Spearheading the global rebranding of OFX

Read more

Thank you so much for sharing this article.Top Digital Marketing company in Bangalore

Way To DM

Predictions: 17 digital marketing trends for 2017

Read more

Thanks for the great article Jodie, agree many boards and senior execs are operating in outdated modes, just as we need some reverse soci...

sharyn

Change across the board: Why boards need to digitally evolve - Data-driven marketing - CMO Australia

Read more

Latest Podcast

More podcasts

Sign in