Five lessons from Expedia’s test-and-learn culture

Global president and CEO shares details the culture and operational strategies allowing the online travel to rapidly innovate

For online travel company, Expedia, there is no good or bad idea, you just need to test them.

During a flying visit to Sydney to mark the company’s 10th birthday in Australia, as well as new plans for the Wotif brand acquired last year, president and CEO, Dara Khosrowshahi, revealed how the group is fostering a culture of rapid innovation.

At any one point in time, brand Expedia is working on 50 different versions of the website and providing daily updates to its customers. Its engineering teams can detect small changes and figure out what version a consumer really likes using data insights, then adapt quickly to suit.

“We don’t have to guess, just test,” Khosrowshahi said.

“What we find is one-third of ideas work, one-third are things the consumer doesn’t care about, and one-third fail. The difference between failure and success can be so small. You need to harness audiences that over time will show if you it’s a good or bad feature.”

Here are five lessons CMO took away from Expedia’s test-and-learn approach that every brand can benefit from.

1. Failure is inevitable, so be transparent about it

According to Khosrowshahi, two-thirds of the things Expedia tests don’t work. And that’s quite alright.

“We embrace a culture that accepts failure as we are embracing failure all the time. It’s about speed and innovation,” he said. “Every time we have a failure, we tell everyone about it across the leadership team. We also drive a culture of transparency.”

Helping foster this openness is a monthly ‘product day’, where each team is invited to present on their past 30 days and next 30 days, and encouraged to discuss both wins and failures.

“Failure has to be part of your culture, then you free people up to take shots,” Khosrowshahi said.

2. Iteration is inherent in innovation

Innovation at Expedia is about constantly iterating its products and features. As an example, Khosrowshahi pointed to its Scratchpad offering, which is now available to consumers across on any device. This allows them to store and edit their holiday ideas and planning, as well as share these with people and communities for interactive discussion.

The feature base has been built up over time, Khosrowshahi said, using data-driven customer insight.

“Scratchpad started as a way for a user to look at their history, then it became something available across all devices, then the editing came in, then it allowed you to send a curated list to friends,” he explained. “Change is step by step.”

3. Ideas can come from anybody

Supporting Expedia’s test-and-learn culture is the concept that ideas can come from any part of its business.

“Regardless of how senior you are, if you have an idea, it will get tested and live or die based on that,” Khosrowshahi said. “Previously, a senior person would make that decision. If that happens, staff will spend all their energy on becoming more senior so they can make decisions. But with engineers, you want them coding, not managing people.

“Your idea is as good as my idea, and that’s been a major cultural change with the company. That has evolved over time.”

Ideas are pooled using a general email address, and product owners with responsibility for different parts of Expedia’s sites can access those and choose which ones to test.

“But overall power is dispersed, and there is that transparency too,” Khosrowshahi said.

“Once per month, we can talk about what you have developed in the last 30 days, where the backlog is and plans for the next 30 days. All ideas and presented and discussed and the whole team votes on your pitch.”

4. Don’t be afraid to retest and mix things up

Expedia is constantly A/B testing, but the measurements and objectives driving these tests vary. One example could be testing features for conversion versus satisfaction.

Alongside digital testing, Expedia maintains a consumer lab, which investigates a host of theories and features relating to its products and services, as well as consumer behaviour. One recent test was using electropads on consumers’ faces to detect microgestures as they interacted with Expedia’s digital offering.

“There’s no good or bad idea, need to test them,” Khosrowshahi added.

Wotif Group managing director, David Finch, said the company isn’t afraid to retest three or four times, or to extend the time spent on a test if it’s struggling to get enough data to verify whether a feature change has been successful.

“The challenge is to get a strong enough signal on what we call a ‘beat, meet or lose’,” he said. “Depending on the size of test, we could test for a week, or just days. If the signal’s not clear, we’ll keep trying.”

Tests are also evoked based around different paths and journeys of a customer, Finch said. But he agreed it’s important not to alienate the customer by introducing too many changes at once.

“For example, we won’t update the landing page for three months at a time. It’s about balance – that’s always top of mind,” Finch said. For instance, the group is keeping the Wotif platform stable at the moment after moving the whole site to the Expedia technology platform a few months ago, he said.

5. Testing isn’t just about technology development

While the test-and-learn approach may have started with websites and A/B testing of technology feature sets, it’s now expanded across all parts of the business, Finch said.

“Our marketing teams are looking to test and learn on brand messaging,” he said. “For example, we will run five versions of an ad, then measure site traffic when an ad is on air, and the one creating the biggest bump will be seen as the best creative.

“Rather than one person deciding on what is best, it’s all based on test and learn.”

More on fostering innovation:

Follow CMO on Twitter: @CMOAustralia, take part in the CMO conversation on LinkedIn: CMO ANZ, join us on Facebook: https://www.facebook.com/CMOAustralia, or check us out on Google+: google.com/+CmoAu

Join the newsletter!

Or

Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.
Show Comments

Latest Videos

Launch marketing council Episode 5: Retailer and supplier

In our fifth and final episode, we delve into the relationship between retailer and supplier and how it drives and influences launch marketing strategies and success. To do that, we’re joined by Campbell Davies, group general manager of Associated Retailers Limited, and Kristin Viccars, marketing director A/NZ, Apex Tool Group. Also featured are Five by Five Global managing director, Matt Lawton, and CMO’s Nadia Cameron.

More Videos

Great read. I agree that it should be a perfect balance between interacting with your customers and knowing your brand. As a business, yo...

Caroline Scott

7 ways CMOs can improve their customer engagement game

Read more

Very true. Team development helps improve collaboration among the team members. I was able to improve my team's collaboration skills by t...

Quent Sinder

Why empowering others can help make you a great leader

Read more

CRM is a very good software that can help you succeed in your business. In my company, this system has allowed me to improve customer rel...

Anna Janicka

Sensis rebrands to Thryv and brings business software to Australian SMBs

Read more

AI Leasing Assistants have finally arrived for the multifamily industry. With so many to choose from it can be hard to figure out which i...

Alice Labs Pte. Ltd.

CMO's top 8 martech stories for the week - 6 May 2021

Read more

Nowadays, when everything is being done online, it is good to know that someone is trying to make an improvement. As a company, you are o...

Marcus

10 lessons Telstra has learnt through its T22 transformation

Read more

Blog Posts

Why if marketing is all you do, you’ll never be very good at it

OK, so you’re probably thinking: “Here comes another article to badger me about living in my bubble.” And also, “I bet this bubble-bashing piece will go on to explain how I can achieve better results through some heady dose of new life experiences, new routines and annoyingly different opinions on social media.”

Dane Smith and Toby Harrison

Ogilvy Australia

A leader’s role in rebuilding a culture of confidence

Every day, there are new predictions and studies on the future of work, the state of the economy and the unfolding global pandemic. All of which creates uncertainty and heightens the imperative of effective leadership.

Michelle Gibbings

Workplace expert, author

Confused About Your Customers?​

​I've worked in brand and marketing for more than 20 years. But there’s one area where I’ve found myself going around in circles and I must admit I'm becoming increasingly confused.

Rich Curtis

CEO, FutureBrand A/NZ

Sign in