Designing AI Systems With Human-Machine Teams

The greatest potential from artificial intelligence will come from tapping into the opportunities for mutual learning between people and machines.

Reading Time: 10 min 



An MIT SMR initiative exploring how technology is reshaping the practice of management.
More in this series
Permissions and PDF

Artificial intelligence promises to augment human capabilities and reshape companies, yet many organizations are finding that the results are falling far short of their expectations. This is frustrating but not surprising. Too often, companies try to implement AI without having a clear understanding of how the technology will interface with people.1

Over the past decade, we have done a number of studies to examine how companies use digital capabilities to become more competitive, including a recent study on human-machine collaboration in a cross-industry setting, where we sought to better understand the contexts in which organizations use particular digital systems.2 In this research, which included more than 20 case studies, we found that many organizations underestimated the value of teaming the predictive capabilities of algorithms with the expertise and intuitions of humans, especially in decision-framing. This led to unsuccessful applications or missed opportunities to learn.

Finding the right balance between machines and people can be extremely challenging even for the most tech-savvy companies. Intel, for example, is seeking to use AI to assess and manage its relationships with more than 19,000 suppliers.3 Intel’s objective is to leverage massive amounts of data in order to select and monitor suppliers to build a more efficient and responsive supply chain. The specific applications it is trying to include in the program are varied. Some tools can help find potential suppliers, while others monitor supply disruptions and reputational risks.

Given its huge data-processing capacity, Intel expects to be able to handle most of the analysis with ease. However, some of the supplier assessments and sourcing decisions will be more complicated, requiring more than the formal rationality that underpins AI algorithms; these will necessitate using the substantive rationality of humans.4 The human input will be particularly critical in situations where it’s necessary to make complex decisions quickly or the signals are weak (for example, when assessing a supplier’s trustworthiness or determining what to expect from a strategic agreement). For these reasons, the system relies on both human and AI-driven elements.



An MIT SMR initiative exploring how technology is reshaping the practice of management.
More in this series


1. D. Lindebaum, M. Vesa, and F. den Hond, “Insights From ‘The Machine Stops’ to Better Understand Rational Assumptions in Algorithmic Decision-Making and Its Implications for Organizations,” Academy of Management Review 45, no. 1 (January 2020): 247-263.

2. M.J. Saenz, J. Rice, and K. Cottrill, “Do You Have the DT’s? Why It’s Time to Bring Digital Transformation Down to Earth,” Supply Chain Management Review, April 1, 2019,

3. “Intel Embarks on a Digital Supply Chain Journey,” SupplyChainBrain, Dec. 19, 2018,

4. S. Barro and T.H. Davenport, “People and Machines: Partners in Innovation,” MIT Sloan Management Review 60, no. 4 (summer 2019): 22-28.

5. “GreyOrange Expands Its Global Footprint, Opens U.S. Headquarters in Atlanta,” Business Wire, Aug. 28, 2018,

6. D. Lee, “Amazon to Deliver by Drone ‘Within Months,’” BBC News Service, June 5, 2019,

7. Lee, “Amazon.”

8. D. Takahashi, “Qoints Uses IBM’s Watson AI to Find Your Influencers,” Venture Beat, May 3, 2018,

9. K. Loria, “Putting the AI in Radiology,” Radiology Today 19, no. 1 (January 2018): 10.

Reprint #:


More Like This

Add a comment

You must to post a comment.

First time here? Sign up for a free account: Comment on articles and get access to many more articles.