Are AI Learning Scenarios Unpredictable Enough?

If AI algorithms are to respond effectively in real-world situations, developers need to consider humanity’s darker impulses.

Reading Time: 4 min 

Topics

Artificial Intelligence and Business Strategy

The Artificial Intelligence and Business Strategy initiative explores the growing use of artificial intelligence in the business landscape. The exploration looks specifically at how AI is affecting the development and execution of strategy in organizations.

In collaboration with

BCG
More in this series

A fender bender heard around the AI world happened last week in Las Vegas when a self-driving shuttle was involved in a minor collision during its first hour of service. It is ironic that this happened in Vegas, a city based on games. How would you score this match between humans and machines? Is it 1-0 in favor of the humans, a victory for the home team?

Not so fast.

In the aftermath of the “calamity,” sensational headlines played to our default thinking that the machine was to blame. Perhaps we humans reveled a tiny bit in the schadenfreude of seeing our emerging computer overlords beaten so quickly when practice was over and the real game started.

But in this incident, the autonomous electric vehicle was shuttling eight passengers around Las Vegas’ Fremont East entertainment district when a human-operated delivery truck backed into the front bumper of the shuttle. Recognizing the oncoming truck, the shuttle stopped to avoid an accident. The human driving the truck, however, did not stop. We instead need to score this matchup as 0-1 in favor of AI.

Worse, this accident illustrates a crucial challenge in the interplay between AI and humans. Systems are typically configured in contexts without nefarious actors, where players are instead well-intentioned and follow the rules. After all, the first step is to get something working.

How does design for the “well-intentioned” manifest itself here? Consider how the situation unfolded: The shuttle seems to have accurately recognized the situation and predicted an imminent collision. This is a current strength of AI — processing input from many signals quickly to build an accurate short-term estimate of what will happen.

Given the prediction, the next step was more difficult. Should the shuttle have honked? That seems fairly risk-free. Reversed and backed away from the approaching truck? That seems more difficult and riskier than a honk. In this case, the shuttle stopped and did nothing — when in doubt, first do no harm. For imperfect AI, faced with uncertainty, a reasonable default is to stop and do nothing.

But this incident should show businesses that thinking about well-intentioned actors won’t be enough. The

Topics

Artificial Intelligence and Business Strategy

The Artificial Intelligence and Business Strategy initiative explores the growing use of artificial intelligence in the business landscape. The exploration looks specifically at how AI is affecting the development and execution of strategy in organizations.

In collaboration with

BCG
More in this series

More Like This

Add a comment

You must to post a comment.

First time here? Sign up for a free account: Comment on articles and get access to many more articles.

Comments (4)
Shrikant Navelkar
Interesting and correct viewpoint. 
Humans and AI are likely to share space for a long long time. It is important to understand where AI is likely to fail or will be hacked and take counter steps. 
The article also highlights how human "thinking" is different from AI "programming". It will be interesting to see how we control human and AI interplay for the benefit of humankind.
Dante Rossi
Hi Mr, Sam!

Congratulations for the article is a very nice point of view, its very interest when we think about how AI could be cheated when is convenient for humans.
I believe that we need embrace AI as part of our life and maybe in some cases thinking in how made error proof systems against humans behavior, for example, create roads and avenues that do not allow the passage of people by the way, only by tunnels and walkways, like science fiction movies.

Thanks for sharing your thoughts with us!
Saradhi Motamarri
Interesting discussion. Part of these elements are in my mind when I characterized what computer is in my article, 'A Diagnostic View On Information Technology,' published in ACM Software Engineering Notes, 17(4), Oct 1992. I observed in that discussion that "A computer is no more powerful than its user".

22 years later to that article, I presented some additional thoughts in a paper, 'Reflections on Artificial Intelligence - A Hermeneutic Journey', presented at ACIS 2014 (Australasian Conference on Information Systems) held in Auckland, New Zealand.
Wesley Alves Machado
Hi, Mr. Sam.
I'd like to congratulation you for the article. I really bring one interesting point of view to this so fresh subject.
It's very important to explore as much dimension as possible when we're talking about new techs.

Regards,

Wesley Alves Machado