Why we need to look inside the black box of AI

How artificial intelligence engines make decisions is just one of a number of logistic and ethical reasons we need to be investigating systems more closely, says this Stanford University expert

In the late 18th century, Wolfgang von Kempelen amazed the world with his chess playing automaton, the Mechanical Turk, as it defeated worthy opponents including Napoleon Bonaparte and Benjamin Franklin.

Onlookers were amazed at how a device of gears and sprockets had been able to master the complexities of chess and demanded to know what was inside.

As it turned out, the chassis of the Mechanical Turk actually housed a human being – a simple yet highly-effective deceit made possible by the inability of onlookers to take a peek inside.

While the story of the Mechanical Turk is more than 100 years old, it may be a useful parable for modern society’s increasing use of artificial intelligence (AI), and how we are often relying on findings with little understanding of how they are reached.

The opacity of AI systems has significant potential for abuse, particularly through the long-term impact of intentional or even unintentional biases in the data. It is a topic that has also become prominent in thinking of many computer scientists and researchers, and given rise to the concept of Explainable AI.

One of the advocates for the potential of Explainable AI is Stanford University’s Executive Director of Strategic Research Initiatives in Computer Science, Dr Stephen Eglash.

Speaking to CMO ahead of his appearance at the Melbourne Business Analytics Conference on 19 July, Dr Eglash described how we have created AI systems that are effectively ‘black boxes’ whose results and decisions can be upsetting to those they impact when we have no idea of how those results are produced.

“If an AI system is helping a loan officer at a bank determine whether you are eligible for a loan, and it decides to reject your loan application, it’s reasonable to expect an explanation,” Dr Eglash says.

The loan applicant Dr Eglash describes may have no idea how the system came to reject their application – or whether it might have been working from data or an algorithm that was somehow prejudiced against the applicant.

“So explainable AI refers to a whole group of different approaches that people are taking to peel back the curtain on that and understand the inner workings of these AI systems,” Dr Eglash says.

One of these methods is to test AIs using different inputs to determine how it is making decisions. A simple example is an AI that has been designed to recognise images of cats. Recent research shows that by feeding that AI partial or doctored images it is possible to determine what parts of the image correspond to “cat” and the point at which the AI’s ability to recognise cats fails. Dr Eglash says this process might also make AIs more resilient, by uncovering the circumstances under which they might fail to make the best decisions.

Bringing in an ethical perspective

In his role at Stanford University Dr Eglash meets with as many as 200 companies and other organisations each year, and says he is pleased many of these conversations now include some reference to the ethics of emerging technology.

“They appear to believe that a company’s reputation for integrity matters,” Dr Eglash says. “And they increasingly seem to believe society’s values ought to become their corporate values as well and avoid going down the path of certain companies that have gotten in trouble for various sorts of bad behaviour or poor values.”

He points out self-interest can be a useful driver of these conclusions, such as through wanting to attract and retain younger workers or to send a positive message to customers.

“Whatever the motivations, I definitely have a sense from the companies that I meet with that we are hearing a lot more about humanitarian and social concerns,” Dr Eglash says.

Hence while the current era represents an incredibly exciting time for marketing and customer service, Dr Eglash says marketers shouldn’t leave their integrity at the front door when they arrive at work.

“It needs to become a part of who you are and a part of everything you and your company does,” Dr Eglash says. “Empathy goes a long way. Respect for the individual goes a long way. Just because you have the data doesn’t give you unlimited rights to do things with it, and as you judge what is appropriate and inappropriate, look at it not only through you own lens but also how it is going to appear to others.”

Follow CMO on Twitter: @CMOAustralia, take part in the CMO conversation on LinkedIn: CMO ANZ, join us on Facebook: https://www.facebook.com/CMOAustralia, or check us out on Google+:google.com/+CmoAu

 

 

 

Join the newsletter!

Or

Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.
Show Comments

Blog Posts

Building a human-curated brand

If the FANG (Facebook, Amazon, Netflix, Google) sector and their measured worth are the final argument for the successful 21st Century model, then they are beyond reproach. Fine-tuning masses of algorithms to reduce human touchpoints and deliver wild returns to investors—all with workforces infinitesimally small compared to the giants of the 20th Century—has been proven out.

Will Smith

Co-founder and head of new markets, The Plum Guide

Sustainability trends brands can expect in 2020

​Marketers have made strides this year in sustainability with the number of brands rallying behind the Not Business As Usual alliance for action against climate change being a sign of the times. While sustainability efforts have gained momentum this year, 2020 is shaping up to be the year brands are really held accountable for their work in this area.

Ben King

CSR manager & sustainability expert, Finder

The trouble with Scotty from Marketing

As a Marketer, the ‘Scotty from Marketing’ meme troubles me.

Natalie Robinson

Director of marketing and communications, Melbourne Polytechnic

If you think it can benefit both consumer and seller then it would be great

Simon Bird

Why Ford is counting on the Internet of Things to drive customer engagement

Read more

It's a good idea. Customers really should control their data. Now I understand why it's important.

Elvin Huntsberry

Salesforce CMO: Modern marketers have an obligation to give customers control of their data

Read more

Instagram changes algorithms every time you get used to them. It really pisses me off. What else pisses me off? The fact that Instagram d...

Nickwood

Instagram loses the like in Australia; industry reacts positively

Read more

I tried www.analisa.io to see my Instagram Insight

Dina Rahmawati

7 marketing technology predictions for 2016

Read more

The saying is pretty tongue in cheek. It's not saying that marketers are bad people, nor that they don't take themselves seriously. There...

LYF Solutions

The trouble with Scotty from Marketing - The CMO view - CMO Australia

Read more

Latest Podcast

More podcasts

Sign in