Topics
Artificial Intelligence and Business Strategy
In collaboration with
BCG
In collaboration with
Harvard Business School professor Raffaella Sadun’s research has historically focused on digital reskilling. Now, rapid technological changes — like AI — are reshaping the nature of work. Raffaella’s research has explored how AI might empower those with intermediate expertise, such as store managers and blue-collar workers, to become more efficient and satisfied in their roles. She shares a bit about her research on today’s episode of Me, Myself, and AI and highlights the potential for AI to improve teamwork and innovation by bridging the gaps between different functional teams. Her experiments show that AI can enhance productivity and output quality, sometimes even substituting for team collaboration, while also improving speed and efficiency in problem-solving tasks.
Subscribe to Me, Myself, and AI on Apple Podcasts or Spotify.
Transcript
Sam Ransbotham: Stay tuned after the episode as Shervin and I discuss the key points from today’s guest.
We hear a lot about augmenting humans with AI, but what if a cross-functional team were 100% AI? Would companies be more innovative? One researcher shares her conclusions on today’s episode.
Raffaella Sadun: I’m Raffaella Sadun from Harvard Business School. You’re listening to Me, Myself, and AI.
Sam Ransbotham: Welcome to Me, Myself, and AI, a podcast on artificial intelligence in business. Each episode, we introduce you to someone innovating with AI. I’m Sam Ransbotham, professor of analytics at Boston College. I’m also the AI and business strategy guest editor at MIT Sloan Management Review.
Shervin Khodabandeh: And I’m Shervin Khodabandeh, senior partner with BCG and one of the leaders of our AI business. Together, MIT SMR and BCG have been researching and publishing on AI since 2017, interviewing hundreds of practitioners and surveying thousands of companies on what it takes to build and to deploy and scale AI capabilities, and really transform the way organizations operate.
Hi, everyone. Thanks for joining us today. We are speaking with Raffaella Sadun, a professor of business administration at Harvard Business School. She’s also cochair of Harvard Business School’s Project on Managing the Future of Work and the coprincipal investigator of the Digital Reskilling Lab. Raffaella, welcome to the show.
Raffaella Sadun: Thank you for having me. It’s my pleasure to be here.
Shervin Khodabandeh: It’s wonderful to have you. Raffaella, you have a long history of researching and publishing on innovation, technology, AI, and the impact on organizations and the nature of work. And that last piece — organizations and nature of work — is actually one of the most important ones in the work that Sam and I have been doing and talking about quite a lot. We see that most companies struggle to get that part right, and that’s where the real hidden value is in getting [the] full impact from AI. Share with us what you’re seeing in your own work and … some of the key highlights.
Raffaella Sadun: Let me start by saying I am doing research on AI, but my ideas on AI are also shaped by research that I did prior to the AI era, more generally on technological change and adaptation [and] how firms adopt new technologies in their organizations.
I think a lot of the attention now is focused on the technological layer or the data layer because it’s clear that these are essential prerequisites for AI to work. You need to know and you need to have enough information to be able to start a prediction process, which is most of what these technologies do. And you want to have enough of a technological layer to be able to interpret and understand what the algorithm is doing for you. I take that for granted.
What I’m observing in organizations is that the organization and the change management part is as important and often neglected. So where do we start? Well, first, you need to understand, “What is the business application?”
We know from the latest surveys a very large number of U.S. workers are engaging with these technologies. But when it comes to the adoption of these technologies in the firm, you first have to ask yourself, “Why am I doing it? What’s the value add?”
That in itself, I think, is one instance where I see tremendous heterogeneity across organizations. Just to be able to ask that question and answer that question, you have to have in mind how your organization adds value relative to [the] competition. And so you want to understand … your competitive strength and how this technology may affect your competitive strength.
Shervin Khodabandeh: Strategy with and for AI. Exactly.
Raffaella Sadun: Exactly. And this is where I come to my second point. My sense is that we are at a point where it is similar to the onset of a new technology paradigm, where a lot of the knowledge about how this technology adds value, and whether it adds value to specific verticals or specific businesses, has not yet been codified.
So this is a time in which there is tremendous return from experimentation and, in particular, tailored experimentation within the firm. And this is where I see organizations diverge. Are you able as an organization to formulate a hypothesis to set up the experimental context that would allow you to test that hypothesis? But also, are you able to codify the learnings that are coming from your experiment and feed it back into the organization?
I don’t see a ton of attention on this step, but I think that this step is really, really, really important for the adoption process.
And this is the third part: At the end of the day, your massive change management task is to convince people to do things differently than what they’re doing now.
Shervin Khodabandeh: It’s a double-edged sword, right? In some ways, you can argue that if you go back 10 years ago [when] AI was doing what AI could do, and only AI could do, which was [processing] massive amounts of data and seeing trends and predicting and optimizing, and humans [were] doing things humans could do, and only humans could do, which is sort of smooth around the edges, consider the strategic options, be creative, negotiate — I would even go as far as saying show empathy — those kinds of things, you’d say, “Yeah, there’s a human and AI, and they each do their own pieces.” And I think what we’re also seeing now is AI is doing things that humans could do and sometimes is doing it better.
So I’d imagine the speed with which organizations have to evolve and change has also drastically changed, and what concerns me is that the rate of change for those who are doing this well and with momentum [is] creating a real gap in every sector. How do you see that evolving?
Raffaella Sadun: I think that it’s helpful to look back at what happened with the ICT [information and communication technologies] revolution, because there is an analogy there with the introduction of computers and software. In the ’90s these technologies were changing the task content of jobs, right?
So we were having some tasks that were being replaced by technology and some tasks that were going to be complemented. Now, this is what happens on average in the economy, but then going back to your point, when you observe how this technological change ended up affecting firms, what you observe in the data — actually, my first paper of my doctoral dissertation was all about this.
Why is it that ICT capital, at that time, was having a very positive effect in some firms and actually not so much in other firms? Well, it all came down to their ability to adopt this technology, which in turn came down to their ability to reallocate tasks or hire different types of workers or change the ways in which work was organized. Because some firms remained behind and … were never able to really keep up with this speed of adaptation on the organizational side — we’re not talking about reinventing the technology, we’re really talking about making sure that the technology works for you — over time, we have seen a massive divergence, at least in terms of productivity, between winners and losers. And I think that there is data out there that traces back [to] the emergence of superstar firms and this disparity in outcomes across firms at the macro level. So we’re really thinking this is having a big impact on the economy [and] the differential ability to implement new technologies.
I think I agree with you. I mean, if history repeats itself, I think that we might see … this further divergence across firms, which is going to be influenced by their ability to experiment and adopt.
Sam Ransbotham: That divergence seems scary though. It speaks to the Matthew Effect, where the rich get richer, the big companies get bigger, the small companies get smaller. How do we keep that from exacerbating inequality?
Raffaella Sadun: My sense, looking at AI adoption right now, it’s very preliminary evidence, but my sense is that precisely because these technologies require a very different organizational setup, it might well be that a new organization that is born today could be better equipped at using this technology and getting exponential returns from these technologies than organizations that have to retrofit what they do to adopt the technology.
My worry is more about which types of workers are going to be able to adapt to these … changes. That’s why I’m so interested in how do you reskill the population? And in particular, maybe because I am becoming a middle-aged person, you know, how do you convince an adult, somebody who has been working for a long time, to change their way of doing things and maybe switching to new occupations or changing what they’re doing? I think that’s actually a very, very tough problem.
Shervin Khodabandeh: Compared to prior milestones or landmarks in technological evolution, do you think that the extent of change in the workforce and reskilling is higher, lower, or similar?
Raffaella Sadun: My sense is that the interest of firms in reskilling is higher, and I attribute this heightened interest to a couple of things. The first one is after COVID’s tightening [of] labor markets really scared companies, they started thinking about reskilling in a different way.
You might say that, you know, labor markets have almost gone back to normal, so that is probably not so much of an incentive right now, but there is one thing that will keep happening, and is happening, which is we’re all getting older. And, in some parts of the world, the aging of the population coupled with the need to adopt new technologies is a real concern.
We have a lab here. It’s called the Digital Reskilling Lab, and we’re working with 15 companies now, some of them quite large, some in Europe, some in Latin America, some in the U.S. And they’ve invested massive resources for these purposes.
Sam Ransbotham: The pace seems difficult here because, I think some of your research pointed out, the average half-life of skills is now less than five years, and in some fields was less than two and a half years. … How do we get off the hamster wheel where we’re retraining just in time for the new technology to change and, again, [constantly] reskilling, never able to use those skills?
Raffaella Sadun: Look, I don’t think that we get off the wheel. Perhaps this is why we are hearing more and more about lifelong learning as being an idea. … I agree with you, part of this change is scary. Do you have to reinvent yourself every so years?
So I don’t think that that’s something that you get out of. That’s where I think AI is particularly interesting. There was a recent report from the National Academies, and they were speculating on what’s the impact of AI on the labor force.
I remember there is this very interesting graph. I think it’s entirely speculative. I don’t think anybody has tested it yet, but think about an inverted U. On the x-axis is the level of expertise of workers, and on the y-axis, it’s the potential returns from AI. The report was speculating that AI would probably have very limited effects on the super naive and less expert workers because they don’t have the contextual knowledge to really use these technologies in the most productive way. People who know something about the context, they have some basic knowledge but they’re not the super experts. The reason why I think this is interesting [is] because it’s true that the costs for those types of workers are quite high. You know, if you ask a person [who] has already started on an occupational pathway to change, this is costly.
We’re doing some studies with companies that are introducing AI for blue-collar workers. At the beginning of the introduction, they started with a lot of concerns about the technology and concerns of what the technology could do to their jobs and whether they would be comfortable using it.
We are halfway through the study, and it seems like job satisfaction is increasing. They are experimenting with the technology — these are blue-collar workers who are using technology to solve problems on machines.
I think it’s too early on to see whether this is a one-off and whether the effects will be permanent. Maybe it’s a transition period. But I definitely find that specific aspect of AI very interesting and different from past technological waves.
What we are talking about here is something that could potentially empower [these] middle expertise workers rather than replacing them. So, for example, how can this technology change the job of a store manager? Could you decentralize more? Could you give them more power and make the job more interesting?
I think we are in a stage in which we don’t know yet whether this is true, but I think it’s an interesting possibility, and that’s why I’m studying it. If that turns out to be feasible and economically productive, I think this has implications for how the technology affects work. I want to go back to Sam’s initial concerns about inequality. This is one way through which, potentially, you could counter these concerns.
Sam Ransbotham: I like the idea of a store manager to make that concrete. This is a particular person that if there are lots of instances of store managers across the nation who could then learn from each other in a way that they’ve never been able to … before, you can see the appeal readily right there versus a replacement argument.
You’ve recently had an experiment about compositions of teams, where some parts of [the] teams are artificial intelligence and some parts of the teams are traditional humans. Tell us about that experiment and what you’re learning.
Raffaella Sadun: This is really going back to understanding how AI can affect the nature of work. As you know, especially in knowledge organizations, work today is organized in teams. You have different functions, different experts coming together to produce.
In particular, in this experiment we have R&D and marketing. So we work with one company. This is a consumer packaging company. Innovation is pretty central to what they do, and their concern is that the way in which this innovation, especially in the early stages, was organized was not optimal because they always needed to have teams — R&D and marketing — to work together.
They were concerned that it was hard to get people together in the same room at a high frequency, but also they were concerned that perhaps there was a little bit of inhibition — the R&D people going crazy, the marketing people telling them we have to make money or vice versa.
The experiment that we ran was what if instead of organizing [under] the status quo [of] teams without AI, let us experiment with four conditions.
One is individuals working on their own. The second is an individual working with AI, and the role of AI is to be a persona. … If you’re an R&D person, [it would] be your marketing counterpart. If you’re a marketing person, [it would] be your R&D counterpart, relative to teams without AI, which is the status quo today, and teams with AI.
And then we put them in an experimental setting. We ask them to innovate. In the context of a hackathon, we try to mimic as much as possible what the company does normally. Then we had the outcome of this process evaluated by experts who applied this process to look at the quality of the submissions or the quality of these ideas.
What we found are a couple of interesting things. First of all, AI and individuals — so this is the person with an AI expert — on average … do really well. The quality of their output is, on average, as good as the quality that comes out of teams with AI.
So this is telling you that, at least in this context, the AI potentially substituted for some of this team expertise. And if you look at why that happened — it’s actually one graph that I love in the paper — you see that when people work by themselves without AI, their ideas pretty much mimic their functional expertise.
If you’re an R&D person, you’re going to propose a very technical proposal. If you’re a marketing person, it will be very commercial. And you should imagine two distinct peaks that overlap a little bit but not so much.
When you put an individual with AI, you see the convergence of the ideas that they propose. And so an R&D person becomes more like a team of R&D and marketing, and the marketing person is more of a team of R&D and marketing. And I think that this is fascinating.
In fact, we find that teams with AI are disproportionately better at placing these ideas on the top decile of the submission. So if you care about the averages, this AI/human team is fine. If you care about the exceptional outcomes, you still need teams of people with AI to work together. So there is still hope for humans.
Sam Ransbotham: I hope that translates also into the idea of, like in the universities, we tend to think our class is the most important thing at the university, but deep down there’s a whole lot that goes on in the university educational experience that is not related to what goes on in the classroom. By analogy, that’s what’s going on in organizations and teams as well.
Shervin Khodabandeh: Sam, as you were saying this, as I was listening to Raffaella, my mind went into, what happens in a team or in a discussion? So much of it is around knowledge and understanding, increasing, codifying knowledge. There, AI clearly has a ton of advantages, and particularly with more agentic forms of AI where, as you were talking about [with] R&D and marketing, now you can have different personas and bring in a ton of different personas with different biases — all that.
I wonder whether, in a roundabout way, AI will make us all much more inspiring and motivating and interesting people.
Raffaella Sadun: I hope so. I agree with you that this is a possibility, but [it could also even give] us more time to spend our precious time, which is really a limited resource, [on] things that only we can do.
Sam Ransbotham: In your experiment, was there any difference in the speed?
Raffaella Sadun: So that’s the other thing. The individuals plus AI, and also the teams with AI, were able to, on average, come up with better solutions but also at a fraction of the time. It’s a productivity effect, if you want to read it that way, that I think is also quite interesting.
Sam Ransbotham: That seems huge.
We have a segment where we want to ask you some rapid-fire questions. Just answer the first thing that comes to your mind.
Raffaella Sadun: OK. I’m not good with those, but go ahead.
Sam Ransbotham: What do you see as the biggest opportunity for artificial intelligence right now?
Raffaella Sadun: What excites me is the democratization of expertise, knowledge.
Sam Ransbotham: What’s the biggest misconception that people have about AI?
Raffaella Sadun: That it’s only about the technology.
Sam Ransbotham: What was the first career that you wanted?
Raffaella Sadun: My first career? I’ve always wanted to be an economist. I know it’s very sad, but [I’ve wanted to] since I was 12.
Sam Ransbotham: Did they have the Fisher-Price little economist set?
Raffaella Sadun: They had the economist Barbie. I don’t know.
Sam Ransbotham: When is there too much artificial intelligence?
Raffaella Sadun: I think when it’s stupid. And this is a real risk, that it says things that look right, and they’re just stupid, and you can’t stop it.
Sam Ransbotham: What’s one thing you wish that artificial intelligence could do right now that it can’t currently do?
Raffaella Sadun: I think it would be. … That’s a good question. I don’t know the answer to that one.
Shervin Khodabandeh: That’s a good answer.
Sam Ransbotham: Thank you for taking the time to talk with us. I think it’s really interesting to think about these huge macro effects you’re talking about. It could be faster or slower. … There [are] good mechanisms behind each one of these possible reasons going on, and we’re just in a big learning process to figure this out. I think we’re all going to have to figure that out together. Thank you for taking the time to join us.
Raffaella Sadun: Thank you. My pleasure.
Shervin Khodabandeh: Thank you. This was wonderful. It was great.
A really interesting conversation with Raffaella, Sam. I particularly like the discussion on [the] transition period between where we are now with AI and where we will be, as it’s beginning to pervasively change the nature of work. And what people do and how people do it and … the analog to industrial revolution and electric motor and all that. And I like the contrast of, you know, if you go really fast because. …
Sam Ransbotham: We’re building on all this experience?
Shervin Khodabandeh: Exactly. We have all this data we never had. We’ve got this knowledge, and we’ve seen how fast. In fact, we do see that in the tech curves, right? If you look at how long it took computer vision to be better than humans, it took a decade and a half, I think. And then [look at] how long … it took for generative AI to be able to do grade school math. A few months. How long did it take to do complex mathematical proofs? Another few months. We see that happening, but in terms of changing the nature of work?
Then the other spectrum was … it could go very slow because it’s going to encroach on what’s uniquely human.
Sam Ransbotham: Also with a population that’s not been subject to this change before. The idea that it’s no longer them but us is a big deal.
Shervin Khodabandeh: Clearly the economic forces are at work here. But I’ve got to believe there [are] going to be regulatory and social forces at play here, too.
Sam Ransbotham: We’re just not used to things changing that quickly on us. Our moon still goes around once a month. Our sun still goes around once a year. We are sort of locked into a lot of patterns as people that just because the technology can move faster, I’m not sure that we can.
Shervin Khodabandeh: The other thing I liked was the experiments that they’d run where collaboration with humans and AI in two different fields, like R&D and marketing, is as good or better, right? That was the conclusion.
[This] is positive in a sense that you have clearly no conflict — because you can’t really have conflict with AI — but doesn’t it also have the really, really scary effect [of] why do I even need the other side? And why does the other side need me? And then we each go to our own little thing, and AI just becomes the proxy —
Sam Ransbotham: The proxy for all this?
Shervin Khodabandeh: Yeah.
Sam Ransbotham: That it seems a little somehow sad, the vision of everybody working in their little room alone with the machine. That makes even me sad, with my hermit-like tendencies.
Shervin Khodabandeh: Well, then what she said is, “Look, the rate at which AI will do the things that it takes you a long time to do is so much faster that then you get to do the things you want to do,” which is true. But then I think about, in a regular person’s 40-hour-a-week job, like how much of their work is really the thing that they just love to do? So much of the work is the thing they have to do.
Sam Ransbotham: Well, I know you think that the podcast is your favorite part of the week.
Shervin Khodabandeh: Yeah, exactly. I work a lot better with my AI cohost.
Sam Ransbotham: But I think her point, though, [is] not just about that particular idea. They recognize that their study has short-term effects just on an idea that there’s a whole lot that goes on in organizations. Maybe the bigger point here is that we’re all learning about this stuff at the same time. No one’s got the answer of how the human world transitions through this rise of machines. We don’t have the book to look at here. The experiment that she’s doing and that millions of other academics and businesses are doing are all part of that. And we’ll hopefully figure it out.
Shervin Khodabandeh: Thanks for listening today. Next time, Sam and I speak with Linda Yao of Lenovo. Please join us.
Allison Ryder: Thanks for listening to Me, Myself, and AI. Our show is able to continue, in large part, due to listener support. Your streams and downloads make a big difference. If you have a moment, please consider leaving us an Apple Podcasts review or a rating on Spotify. And share our show with others you think might find it interesting and helpful.
