Managing With Immature AI

Many of our current experiences with AI applications so far are contradictory, largely because AI is still a young, immature field. Consider the following:

• Algorithms are competing with, and often winning against, expert humans in complex games such as chess and Go. But at the same time, customer service chatbots often are easily confused and can be more annoying than helpful. These interactions are more reminiscent of the transparent ELIZA than the almost human Nexus-6.
• Hardly a day goes by without more breathless reporting about improvements in self-driving cars. Yet, my own attempts to use AI for far simpler tasks, like scheduling meetings, have been frustrating for all involved — and I don’t yet see a glimmer of hope that my fixed-cost investments in setup will ever be offset in improved processes. I currently can’t convince the AI-based scheduler that Florida is not the best place for a meeting with a colleague located next door in Boston, though perhaps the AI has deeper predictive insights into the weather that it thinks I should factor in.
• AI has the potential to apply data and algorithms to offer quick decision making based on open rubrics, exposing the processes behind decisions that are now made in the shadows behind closed doors. However, attempts so far in contexts such as crime investigation or contest judging might simply be improving speed and reinforcing bias, which is not exactly an area our society needs help increasing.

It can be difficult for managers to reconcile the vision of AI with the reality of its current state. Rather than getting clean, unequivocal answers about the usefulness of AI, managers may need to deal with a somewhat messy fact: Even as it continues to fall short on its promise, AI has tremendous potential.

Facing this contradiction, the temptation may be strong to sit on the sidelines and wait to see how things settle. But I don’t think things will settle, at least not any time soon.

In the MIT SMR article, “Minding the Analytics Gap”, my coauthors and I discussed how analytics advances are outpacing organizational abilities to use those advances. Organizations are able to add complexity to their analytical production capabilities faster than they can consume this increased complexity. In this scenario, the gap between what the organization can produce and what it can consume grows rather than shrinks as organizations build analytical capabilities.

AI has many similarities. As smart people continue to push the boundaries of what’s possible with AI, organizations will likely struggle to incorporate prior advances into repeatable business processes; that is, the promise and reality of AI may diverge instead of converge. So managers must work with imperfect AI now. How are managers pragmatically incorporating AI into current business processes?

• They are savvy about where they apply AI. It may be tempting to think chatbots can replace customer service workers. That certainly does seem profitable. But the net effect is to replace dynamic, situationally appropriate employee-customer interactions with generic, still immature machine — customer interactions, which may do more harm than good in the short term — it’s unlikely that customers, suppliers, and vendors will want to be your organization’s AI guinea pigs. Instead, organizations can use AI to support employees internally so that employees insulate customers or vendors while the AI gains experience. Or organizations can use AI to improve information and processes to reduce the number of customers requiring support.
• They build in checks and balances. It is unreasonable to expect AI to work unmonitored. Just because the “employee” is a machine, managers cannot abdicate management. For example, when early algorithmic pricing algorithms were left unchecked, prices quickly escalated to ludicrous levels. Instead, organizations can recognize the adjustments that AI will need and pay particular attention because errors can now compound extremely quickly. At least in the short term, AI may increase rather than decrease demands on management.
• They test to gain experience in low-stakes environments. Soon after Microsoft launched Tay, an AI-based Twitter account, things got really ugly, really fast. There was lots of news attention to the “failure.” But I don’t think it really had a lasting, negative effect on Microsoft. The company tested this technology in a context with low consequences and learned a great deal in the process. While some of their lessons may have been ones they’d rather forgo, future products in more critical contexts will benefit from the Tay experience.
• They are upfront about their use of AI technology. It takes milliseconds for people to figure out that robocall telemarketers pretending to earnestly inquire about your credit card debt or home security system needs are just interactive voice-response systems. But exactly 0% of people feel really impressed with the telemarketer’s technology that fooled them for those milliseconds and are eager to continue a relationship that starts on duplicitous footing. Instead, organizations that use AI to interact with customers should be up front about it. Just as with data collection, when it comes to using AI, trust is difficult to earn but easy to lose.