Five lessons from Expedia’s test-and-learn culture

Global president and CEO shares details the culture and operational strategies allowing the online travel to rapidly innovate

For online travel company, Expedia, there is no good or bad idea, you just need to test them.

During a flying visit to Sydney to mark the company’s 10th birthday in Australia, as well as new plans for the Wotif brand acquired last year, president and CEO, Dara Khosrowshahi, revealed how the group is fostering a culture of rapid innovation.

At any one point in time, brand Expedia is working on 50 different versions of the website and providing daily updates to its customers. Its engineering teams can detect small changes and figure out what version a consumer really likes using data insights, then adapt quickly to suit.

“We don’t have to guess, just test,” Khosrowshahi said.

“What we find is one-third of ideas work, one-third are things the consumer doesn’t care about, and one-third fail. The difference between failure and success can be so small. You need to harness audiences that over time will show if you it’s a good or bad feature.”

Here are five lessons CMO took away from Expedia’s test-and-learn approach that every brand can benefit from.

1. Failure is inevitable, so be transparent about it

According to Khosrowshahi, two-thirds of the things Expedia tests don’t work. And that’s quite alright.

“We embrace a culture that accepts failure as we are embracing failure all the time. It’s about speed and innovation,” he said. “Every time we have a failure, we tell everyone about it across the leadership team. We also drive a culture of transparency.”

Helping foster this openness is a monthly ‘product day’, where each team is invited to present on their past 30 days and next 30 days, and encouraged to discuss both wins and failures.

“Failure has to be part of your culture, then you free people up to take shots,” Khosrowshahi said.

2. Iteration is inherent in innovation

Innovation at Expedia is about constantly iterating its products and features. As an example, Khosrowshahi pointed to its Scratchpad offering, which is now available to consumers across on any device. This allows them to store and edit their holiday ideas and planning, as well as share these with people and communities for interactive discussion.

The feature base has been built up over time, Khosrowshahi said, using data-driven customer insight.

“Scratchpad started as a way for a user to look at their history, then it became something available across all devices, then the editing came in, then it allowed you to send a curated list to friends,” he explained. “Change is step by step.”

3. Ideas can come from anybody

Supporting Expedia’s test-and-learn culture is the concept that ideas can come from any part of its business.

“Regardless of how senior you are, if you have an idea, it will get tested and live or die based on that,” Khosrowshahi said. “Previously, a senior person would make that decision. If that happens, staff will spend all their energy on becoming more senior so they can make decisions. But with engineers, you want them coding, not managing people.

“Your idea is as good as my idea, and that’s been a major cultural change with the company. That has evolved over time.”

Ideas are pooled using a general email address, and product owners with responsibility for different parts of Expedia’s sites can access those and choose which ones to test.

“But overall power is dispersed, and there is that transparency too,” Khosrowshahi said.

“Once per month, we can talk about what you have developed in the last 30 days, where the backlog is and plans for the next 30 days. All ideas and presented and discussed and the whole team votes on your pitch.”

4. Don’t be afraid to retest and mix things up

Expedia is constantly A/B testing, but the measurements and objectives driving these tests vary. One example could be testing features for conversion versus satisfaction.

Alongside digital testing, Expedia maintains a consumer lab, which investigates a host of theories and features relating to its products and services, as well as consumer behaviour. One recent test was using electropads on consumers’ faces to detect microgestures as they interacted with Expedia’s digital offering.

“There’s no good or bad idea, need to test them,” Khosrowshahi added.

Wotif Group managing director, David Finch, said the company isn’t afraid to retest three or four times, or to extend the time spent on a test if it’s struggling to get enough data to verify whether a feature change has been successful.

“The challenge is to get a strong enough signal on what we call a ‘beat, meet or lose’,” he said. “Depending on the size of test, we could test for a week, or just days. If the signal’s not clear, we’ll keep trying.”

Tests are also evoked based around different paths and journeys of a customer, Finch said. But he agreed it’s important not to alienate the customer by introducing too many changes at once.

“For example, we won’t update the landing page for three months at a time. It’s about balance – that’s always top of mind,” Finch said. For instance, the group is keeping the Wotif platform stable at the moment after moving the whole site to the Expedia technology platform a few months ago, he said.

5. Testing isn’t just about technology development

While the test-and-learn approach may have started with websites and A/B testing of technology feature sets, it’s now expanded across all parts of the business, Finch said.

“Our marketing teams are looking to test and learn on brand messaging,” he said. “For example, we will run five versions of an ad, then measure site traffic when an ad is on air, and the one creating the biggest bump will be seen as the best creative.

“Rather than one person deciding on what is best, it’s all based on test and learn.”

More on fostering innovation:

Follow CMO on Twitter: @CMOAustralia, take part in the CMO conversation on LinkedIn: CMO ANZ, join us on Facebook: https://www.facebook.com/CMOAustralia, or check us out on Google+: google.com/+CmoAu

Join the CMO newsletter!

Error: Please check your email address.
Show Comments

Supporting Association

Blog Posts

Top tips to uncovering consumer insights for business innovation

An in-depth understanding of consumers sits at the heart of what we all need to do, but we know it’s not always easy to uncover insights that will unlock a true innovation opportunity.

Matt Whale

Managing director, How To Impact

Is your customer experience program suffering bright shiny object syndrome?

You may have heard of ‘bright shiny object syndrome’. The term is used to describe new initiatives undertaken by organisations that either lack a strategic approach, or suffer from a failure to effectively implement.

Leveraging technology to stand out in the sea of sameness

The technology I'm talking about here is data and marketing automation. Current digital marketing methodology, much as it is practiced at Bluewolf, dictates the need for a strategy that does four things: Finds the right audience, uses the right channel, delivers the right content, and does all of that at the right time.

Eric Berridge

CEO and co-founder of Bluewolf, an IBM Company

Lead Management is very important part of the process. For anyone running Facebook Lead Ads I would recommend using this service.Get your...

Dirk Lo

How this fintech startup is improving content marketing and lead generation

Read more

I am agreeing with Mr. Tyron Hayes that a measured test-and-learn approach could be missing opportunities to not only better engage custo...

brunson5862@mail.ru

CMO interview: How Curtin University’s marketing chief is using test and learn to cope with complexity

Read more

Excellent!

Dr Sadasivan,US

Shakespeare shows data and creativity aren’t Montagues and Capulets

Read more

Great article! Agreed with all... Matthew Lerner, Deeps De Silva... When a company has a great product that solves customers needs, a gre...

James Tyler

Why marketers are embracing growth hacking techniques

Read more

Very good article, Social media analytics helps in problem identification. They can serve as an early warning system for negative custome...

BizVinu

Four ways to use social media to boost customer loyalty

Read more

Latest Podcast

More podcasts

Sign in