Why bias is the biggest threat to AI development

Panel of artificial intelligence luminaries debate the human and data biases that could turn the opportunities of machine learning into a nightmare for business and human beings

From left: Dr Peter Norvig, Dr Richard Socher, Dr Suchi Saria, and Adam Lashinsky from Fortune magazine
From left: Dr Peter Norvig, Dr Richard Socher, Dr Suchi Saria, and Adam Lashinsky from Fortune magazine

Bias – both human and data-based – is the biggest ethical challenge facing the development and adoption of artificial intelligence, according to a panel of world-leading AI luminaries.  

Speaking at last week’s Dreamforce conference, Salesforce chief scientist and adjunct professor of Stanford’s computer science department, Dr Richard Socher, said the rapid development of AI will inevitably impact more and more people’s lives, raising significant ethical concerns.

“These algorithms can change elections for the worse, or spread misinformation,” he told attendees. “In some benign natural language processing classification algorithms, for example, you may want to maximise the number of clicks, and find something with a terminator image has more clicks so you put more of those pictures in articles.”  

But it is the bias coming through existing datasets being used to train AI algorithms that arguably presents the biggest ethical problem facing industries.

“All of our algorithms are only as good as a trained data we give it,” Dr Socher said. “If your training data has certain biases against gender, age or sexual orientation, it will get picked up.

“Say you are a bank and want to build a loan application classifier to grants loan to new founders of businesses or not. It turns out that in the past, only 5 per cent of approved applications were given out to female founders. The algorithm will pick that up and say it’s a bad to be a female founders, we shouldn’t give out approval. That’s a bad thing to do but it is the past of your dataset.

“Biases have existed; humans have bias. Now that humans have created those data sets, algorithms will have them and potentially amplify them and make them worse.”

Dr Socher noted educational institutions such as Berkeley, as well as tech innovators like Google, have already started investigating how to get rid of bias on the algorithmic side. “But it’s your data set side you have to carefully think about,” he said.

As an example, Dr Socher said his own team had wanted to build an ‘emotion classification’ algorithm so that if an individual entered a physical space, it could identify if they are happy, sad, surprised or grumpy.

“I immediately said we cannot ship this until we look at all protected classes and make sure all old people are not classified as grumpy, for instance, because we only have two stock images of old people showing them happy,” he said. “We have to have some empathy in some ways with how those will be deployed and think carefully about the training data we give it.”

For assistant professor of Stanford University’s Machine Learning and Data Intensive Computing Group and Computational Biology and Medicine Group, Dr Suchi Saria, the rapid industrialisation of machine learning has not only accelerated AI development, it’s brought fresh ethical concerns with it. She saw the challenge as education, rather than technological innovation, but again, one that links back to bias.

“Up until three years ago, we were a small group of AI experts, and we were still kids with toys: No one was monitoring or bugging us, we were coming up with new ideas, and it was all good,” she said. “Things worked and didn’t work.

“Now, we’re in a new place where the tools we are developing and releasing open source, are getting opened up to lots of people. They’re experimenting, putting it in their own world experiences and taking it very seriously sometimes in ways they shouldn’t.”

An example Saria pointed to was using image recognition to try and predict if someone is going to become a criminal or not.  

“The science behind that doesn’t make any sense,” she said. “You can take a large database, then train something in that database and annotate it to provide some supervision, but it’s just mimicking what it has already seen,” she explained. “It’s not doing any causal inference to understand what the mechanism is that makes you a criminal. All it’s doing is replicating behaviour.

“There are tools freely available, there’s engineering experience available to use these tools and they’re becoming easier to use in new domains. But the education on what is a valid or invalid use of these tools is drastically lagging. We suddenly see lots of interesting new applications, but every so often we see applications that are not right; they’re incorrect or biased. They have consequences but there’s no one to police them. As a group, we are used to talking to each other and figuring out what is the right thing to do, and being told when something doesn’t make sense. This industrialisation of AI has changed that.”

Google director of research and revered computer science academic, Dr Peter Norvig, called out for more transparency around how AI is being trained and used.

“With basic AI, we’re collecting data, which means we have to be good shepherds,” he continued. “This isn’t AI per se, but it goes along with it.

“And you always want to have AI embedded in a larger process from which there is a way to ‘escape’. AI shouldn’t be the final link; at some point you need to hit zero and get back to a human operator. These systems are not necessarily good a novel things. It has to be designed to get around that.”  

Norvig pointed out the emerging field of study around ‘AI safety’ needs to be firmly embedded into any AI application.

“We don’t have engineering safety as a separate field, we need it [AI safety] to be embedded everywhere so right from the start you’re aware of what could go wrong,” he said.  

“Software always has bugs and we have tools to try and eliminate that. AI must also use all the best practices that exist in social engineering. Sometimes, however, it’s being driven by academic researchers who aren’t across that software history. But we also have to come up with new tools too.”  

Whatever you do, never think of AI in isolation, Dr Socher added. “You’re always applying it to a specific skill or your business,” he said.

“If you think about what that X is, in most cases getting there means starting with your training data. Think about how to collect output from every business process so when you do bring in AI you have that edge against your competitors. And if you’re working with data scientists and other vendors, they will need a way to access your data and a way for you to get that out.”

Read more of our coverage on the impact of AI

  • Nadia Cameron travelled to Dreamforce as a guest of Salesforce.

Follow CMO on Twitter:@CMOAustralia, take part in the CMO conversation on LinkedIn: CMO ANZ, join us on Facebook: https://www.facebook.com/CMOAustralia, or check us out on Google+:google.com/+CmoAu

Join the newsletter!

Or

Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.
Show Comments
cmo-xs-promo

Latest Videos

More Videos

Nice blog!Blog is really informative , valuable.keep updating us with such amazing blogs.influencer agency in Melbourne

Rajat Kumar

Why flipping Status Quo Bias is the key to B2B marketing success

Read more

good this information are very helpful for millions of peoples customer loyalty Consultant is an important part of every business.

Tom Devid

Report: 4 ways to generate customer loyalty

Read more

Great post, thanks for sharing such a informative content.

CodeWare Limited

APAC software company brings on first VP of growth

Read more

This article highlights Gartner’s latest digital experience platforms report and how they are influencing content operations ecosystems. ...

vikram Roy

Gartner 2022 Digital Experience Platforms reveals leading vendor players

Read more

What about this one FormDesigner.pro? I think it's a great platform providing a lot of options, you can collect different data and work w...

Salvador Lopez

Gartner highlights four content marketing platform players as leaders

Read more

Blog Posts

​Why we need to look at the whole brand puzzle, not just play with the pieces

Creating meaningful brands should be a holistic and considered process. However, all too frequently it’s one that is disparate and reactive, where one objective is prioritized at the expense of all others. So, what are the key pieces to the ‘good’ brand puzzle?

Marketing overseas? 4 ways to make your message stick

Companies encounter a variety of challenges when it comes to marketing overseas. Marketing departments often don’t know much about the business and cultural context of the international audiences they are trying to reach. Sometimes they are also unsure about what kind of marketing they should be doing.

Cynthia Dearin

Author, business strategist, advisor

From unconscious to reflective: What level of data user are you?

Using data is a hot topic right now. Leaders are realising data can no longer just be the responsibility of dedicated analysts or staff with ‘data’ in their title or role description.

Dr Selena Fisk

Data expert, author

Sign in