CMO

7 lessons AGL’s innovation team learnt when building a chatbot

Energy retailer's innovation manager shares the human-centred design thinking and iterative approach behind the group's chatbot


Building a successful use case for emerging technologies like chatbots is less about the technology and more about your ability to embrace human-centred ways of thinking and working, AGL’s innovation manager, Sam King, says.

AGL’s innovation team has spent the past 18 months working on and launching a chatbot. The work was not only about tapping new technology in order to improve customer engagement experiences with the energy retailer, it was also ostensibly a lesson to the rest of the business on the power of human-centred design and iterative innovation.

Speaking at the IBM Think conference in Sydney, King said AGL established a dedicated innovation function in order to better unlock fresh thinking across the organisation. The division prioritises work around three pillars: Capability uplift, which is predominantly about human-centred design; embedding a discipline to encourage behavioural change once teams have exposed themselves to 21 century skillsets; and seeding a mindset and environment where it’s safe to fail, King said.

“While it is nebulous, the third pillar of activity are perhaps the most important,” he told attendees.  

“We also try and incubate how we think things should be done and by tackling high-value business problems that people in other parts of the organisation might be intimated by.”

It’s this human-centred design and iterative approach that’s quintessential when building a chatbot, King said. “It’s not like building regular digital kit. If you go to Web page, the guiderails are pretty clear. With a chatbot, you can’t cater for everything thrown at you,” he explained. “You have to take that into account.”

AGL kicked off work on its chatbot 18 months ago. Over a period of weeks, the innovation team set up chatbot to see what would happen in an approach King labelled “starting before you are ready”. The overarching ambition is solving high-volume business problems, such as transactions, and working backwards.

“We had one senior leader look us in the eye and ask why we would invest in us. Another was in digital and still didn’t think this was a viable, commercial opportunity that would be designed by customers,” King said. “But we had an inkling that things were accelerating rapidly and we needed to experiment.”

Two big lessons came off that experiment. One is that success has nothing to do with the technology, King said.

“It’s really about human-centred innovation. You need to get in front of customer as quickly as possible to ascertain if you are on the right path,” he said. “Your interactions will be different as energy retailer versus bank. So, get in as swiftly as possible to know where to go.”

Related: CMO interview: What AGL’s marketing chief is doing to embrace ambiguity

The second learning was that AGL’s innovation team needed a corresponding command panel.  This mean an operating interface which the customer operations team could sit in front of and manage thousands of chats in real time.

“Chatbots have been done a disservice by Facebook and what other organisations have done in last 15 months,” King claimed. “It’s probably incorrect to think of these as purely autonomous bots. Instead, think of them as semi-autonomous bots which at times requires intervention. Then you’ll see real-world value.”

Following the first prototype, AGL partnered with IBM’s Watson artificial intelligence (AI) platform, as well as LivePerson, its traditional online chat messaging partner, to build the user interface on the Web page as a gateway between the intelligence of Watson and data from AGL. And the live chatbot, Alfie, was launched.

So what lessons did AGL learn?

Lesson 1: Iterate quickly

King said the team iterated quickly, a vital element in embracing emerging technology. As an example of one immediately issue, he pointed to a customer asking the chatbot ‘what it like having an American CEO’.

“We didn’t program for that, so you have to get on to those things as quickly as possible and that means iterating,” he said.  

Lesson 2: Make it easy for humans

The current structure of Alife is to guide people down a particular path around account and transactional inquiries. There are predominantly billing and checking your balance inquiries.

“You need to try and think about how a human would interact,” King said. “For example, no one knows their account number, so we use mobile number as a proxy. That benefits the customer’s experience.”

Another objective was to try to contain the experience in the bot so customers didn’t have to be pushed to Web pages. King said AGL is now in the process of loading 9000 scripts from chat into the back-end to cater for more scenarios via the chatbot.

Lesson 3: Have metrics for success

Throughout the process, King said it was important to have clear metrics for success.

“We doubled our customer satisfaction in one month by reviewing what customers were doing an iterating on a day-to-day basis,” he said. “An end of chat survey is essential to that. Our intent is to get to same level of customer satisfaction as with our human agents, then increase on that.”

Lesson 4: Know when it’s time to go live

Another consideration is the “when” to launch a chatbot for your brand.

“In time, we’ll have generalist bot to answer all questions. Right now, however, we can only cater for limited number of transaction intents from customers,” King said. “So you want to think about when you surface your chatbot – on which page, or how long after they have been browsing.”

Lesson 5: Adapt your testing principles

Thinking differently about testing for the chatbot also proved critical. “We tripped up without automated testing on day one,” King admitted. “We had one comma in the text which broke everything. Save yourself the heartache by having automatic testing from the get go.”

Lesson 6: Embrace personality and personalisation

Personality is a further consideration and potentially something digital and innovation teams haven’t had as much exposure to. According to King, the rule of thumb with a chatbot is to have a tone of voice “that’s witty and informative but not amusing. Comedy is subjective.”

Context is another killer. “Many organisations are thinking more about personalisation, getting to that segment of one, and that’s the opportunity digital can bring to bear,” he said. “When you boot up chat, you should have a lot of information about individuals in the first place, then build on that.”

To help, AGL is looking to tap into Watson’s real-time sentiment longer term so that if chat is in full flight and a red flag is going off, teams can tackle it in the moment.

Lesson 7: Find the tech path of least resistance

In hindsight, if there’s one thing he’d change, King said it would have been going with IBM and LivePerson in the first place. “Find the path of least resistance. LivePerson does 50 million chats per month and has a strategic relationship with Watson. This means they plug in relatively effortlessly and we could build on the shoulders of giants,” he said.  

While AGL’s chatbot is fairly limited at present, King saw a future where customer questions can be answered in different ways.  

“If true listening is understanding how people feel and communicating that, then we do want to get to the point where people feel like the bot is listening,” he concluded. “That might mean your experience with a bot, even if you ask the same questions, might be different to my experience. That’s based on the data we have and what we think will work for you.”

Read more of CMO's coverage of IBM Watson in practice:

Follow CMO on Twitter: @CMOAustralia, take part in the CMO conversation on LinkedIn: CMO ANZ, join us on Facebook: https://www.facebook.com/CMOAustralia, or check us out on Google+:google.com/+CmoAu