AI on the Front Lines
AI progress can stall when end users resist adoption. Developers must think beyond a project’s business benefits and ensure that end users’ workflow concerns are addressed.
Topics
It’s 10 a.m. on a Monday, and Aman, one of the developers of a new artificial intelligence tool, is excited about the technology launching that day. Leaders of Duke University Hospital’s intensive care unit had asked Aman and his colleagues to develop an AI tool to help prevent overcrowding in their unit. Research had shown that patients coming to the hospital with a particular type of heart attack did not require hospitalization in the ICU, and its leaders hoped that an AI tool would help emergency room clinicians identify these patients and refer them to noncritical care. This would both improve quality of care for patients and reduce unnecessary costs.
Aman and his team of cardiologists, data scientists, computer scientists, and project managers had developed an AI tool that made it easy for clinicians to identify these patients. It also inserted language into the patients’ electronic medical records to explain why they did not need to be transferred to the ICU. Finally, after a year of work, the tool was ready for action.
Get Updates on Leading With AI and Data
Get monthly insights on how artificial intelligence impacts your organization and what it means for your company and customers.
Please enter a valid email address
Thank you for signing up
Fast-forward three weeks. The launch of the tool had failed. One ER doctor’s comment that “we don’t need a tool to tell us how to do our job” is typical of front-line employees’ reactions to the introduction of AI decision support tools. Busy clinicians in the fast-paced ER environment objected to the extra work of inputting data into a system outside of their regular workflow — and they resented the intrusion on their domain of expertise by outsiders who they felt had little understanding of ER operations.
Similar failed AI implementations are playing out in other sectors, despite the fact that these new ways of working can help organizations improve product and service quality, reduce costs, and increase revenues.
References
1. S. Pachidi, H. Berends, S. Faraj, et al., “Make Way for the Algorithms: Symbolic Actions and Change in a Regime of Knowing,” Organization Science 32, no. 1 (January-February 2021): 18-41.
2. M. Valentine and R. Hinds, “‘Rolling Up the Leaf Node’ to New Levels of Analysis: How Algorithmic Decision-Making Changes Roles, Hierarchies, and Org Charts,” working paper, Stanford University, Stanford, California, May 2021.
3. E. van den Broek, A. Sergeeva, and M. Huysman, “When the Machine Meets the Expert: An Ethnography of Developing AI for Hiring,” MIS Quarterly 45, no. 3 (September 2021): 1557-1580.
4. T. DeStefano, M. Menietti, and L. Vendraminelli, “A Field Experiment on AI Adoption and Allocation Efficiency,” work in progress.