AI on the Front Lines

AI progress can stall when end users resist adoption. Developers must think beyond a project’s business benefits and ensure that end users’ workflow concerns are addressed.

Reading Time: 21 min 

Topics

Buy

It’s 10 a.m. on a Monday, and Aman, one of the developers of a new artificial intelligence tool, is excited about the technology launching that day. Leaders of Duke University Hospital’s intensive care unit had asked Aman and his colleagues to develop an AI tool to help prevent overcrowding in their unit. Research had shown that patients coming to the hospital with a particular type of heart attack did not require hospitalization in the ICU, and its leaders hoped that an AI tool would help emergency room clinicians identify these patients and refer them to noncritical care. This would both improve quality of care for patients and reduce unnecessary costs.

Aman and his team of cardiologists, data scientists, computer scientists, and project managers had developed an AI tool that made it easy for clinicians to identify these patients. It also inserted language into the patients’ electronic medical records to explain why they did not need to be transferred to the ICU. Finally, after a year of work, the tool was ready for action.

Fast-forward three weeks. The launch of the tool had failed. One ER doctor’s comment that “we don’t need a tool to tell us how to do our job” is typical of front-line employees’ reactions to the introduction of AI decision support tools. Busy clinicians in the fast-paced ER environment objected to the extra work of inputting data into a system outside of their regular workflow — and they resented the intrusion on their domain of expertise by outsiders who they felt had little understanding of ER operations.

Similar failed AI implementations are playing out in other sectors, despite the fact that these new ways of working can help organizations improve product and service quality, reduce costs, and increase revenues.

Topics

References

1. S. Pachidi, H. Berends, S. Faraj, et al., “Make Way for the Algorithms: Symbolic Actions and Change in a Regime of Knowing,” Organization Science 32, no. 1 (January-February 2021): 18-41.

2. M. Valentine and R. Hinds, “‘Rolling Up the Leaf Node’ to New Levels of Analysis: How Algorithmic Decision-Making Changes Roles, Hierarchies, and Org Charts,” working paper, Stanford University, Stanford, California, May 2021.

3. E. van den Broek, A. Sergeeva, and M. Huysman, “When the Machine Meets the Expert: An Ethnography of Developing AI for Hiring,” MIS Quarterly 45, no. 3 (September 2021): 1557-1580.

4. T. DeStefano, M. Menietti, and L. Vendraminelli, “A Field Experiment on AI Adoption and Allocation Efficiency,” work in progress.

Reprint #:

63412

More Like This

Add a comment

You must to post a comment.

First time here? Sign up for a free account: Comment on articles and get access to many more articles.