Topics
Artificial Intelligence and Business Strategy
In collaboration with
BCG
In collaboration with
Over the past year, we’ve seen generative AI explode. In this episode, we review insights shared with us from five prior guests — from Microsoft, GitHub, Meta, Partnership on AI, and NASA — and consider what’s changed, what’s the same, and what new concerns organizations face. With GenAI tools becoming ubiquitous and democratized, organizations grapple with how to use them at the enterprise level and how to regulate their use for employees. They’re also struggling with openness and transparency in the name of knowledge sharing while protecting competitive advantage.
The balance between openness, competition, and responsible deployment of AI is crucial as AI tools continue to evolve.
For more, listen to these prior episodes in full:
- Out of the Lab and Into a Product: Microsoft’s Eric Boyd
- If 10% of the World Were Developers: GitHub’s Mario Rodriguez
- Sharing AI Mistakes: Partnership on AI’s Rebecca Finlay
- Building Connections Through Open Research: Meta’s Joelle Pineau
- AI on Mars: NASA’s Vandi Verma
Subscribe to Me, Myself, and AI on Apple Podcasts or Spotify.
Transcript
Shervin Khodabandeh: AI is quite literally everywhere. Sam and I come on here every other week to talk with smart people about how they use it at work and in their personal lives. In the last year, we’ve heard a lot of predictions and watch-outs, and we’d like to spend some time reflecting on these. What’s changed? What stayed the same? What concerns are still top of mind?
We were fortunate to speak with Microsoft’s Eric Boyd right as ChatGPT-3.5 hit the scene and everyone suddenly became aware of the potential of LLMs [large language models].
Eric Boyd, corporate vice president of AI, Microsoft: One of the things I push my team on a bit is, I want us to think about scenarios and products where the portion of people using AI-powered features is 100%, right? You can think of something like, “Yeah, in this Teams call, we have transcription and so we could turn transcriptions on,” but not everybody uses that. You could go your whole day never using transcription. What are the products [where] you absolutely can’t avoid AI because it’s just intrinsic to the product? Search is, of course, like that. You can’t avoid using AI in search. When you are talking to your phone to compose a text message with speech, you’re using AI 100% of the time you do that.
And so increasingly, as we see these scenarios, there are so many things that are just not possible, right? If I’m going to now start with my three bullet points and have that expanded into a paragraph for me, like, well, you can’t do that without AI. So every time you’re using that functionality, you’re using AI. It’s pretty ubiquitous, [and] there are really a lot more scenarios coming on that. There’s this whole field of AI-powered applications that is really about to start to blossom, where the application just doesn’t exist without the AI that [powers] it.
It’s really kind of crazy just how quickly this field is moving. The speech quality that we deliver through our speech API literally improves every month. We measure, and we’ve got data to back that up. Our vision models have just exploded in quality recently, and we’ve seen lots of crazy things. And then let’s not even get started on language, right? The large language models are just incredibly powerful these days, and so [there’s] just a massive explosion going on there.
Someone was making a joke the other day. They said, “We’re not too far from the days where a CEO is going to have four bullet points and ask an AI to create a two-page memo for his staff, and the staff is going to use an AI to reduce this two-page memo to four bullet points and then go and read them.” I thought that was very funny.
For that type of process, though, I’ve used AI already to say, “I’ve got some tough email I need to write; I want to make sure I’m getting the tone right. Here’s how I wrote it. Can you make this more polite, or can you give me a suggestion on how I should change it?”
Just being able to get valuable input and feedback on that — it’s kind of amazing. That’s really empowering and so very central, then, to the work that you’re going to try and do as a result of that.
We have a lot of scenarios. We’ve got AI embedded into Microsoft Dynamics; it’s a CRM and advertising tool. One of the things we’re using it for is to help people create advertising copy. They can create advertising copy that gets generated for them. A similar use case: We worked with CarMax, and CarMax has every single car on the planet, and they want to have a unique page describing every single car on the planet.
Whenever I talk about CarMax, I say, “Well, my first car was a 1986 Ford Tempo. The 1986 Ford Tempo was a piece of garbage. It was a piece of garbage in 1986, and it’s absolutely one now, but that was my first car.”
So CarMax wants to have a page describing the 1986 Ford Tempo. And they have user reviews about it, but they want a page that describes it, that will do really well for search engine optimization. And so they used GPT-3 to go and summarize reviews that they’ve got on each make, each model, each year, and then generate a page for it. It would have taken them years, literally years — they did the math on how you could sort of go and do that. But now they have this really high-quality, valuable content that’s directing people to their site as a result of that.
San Ransbotham: And what’s exciting about that is the scale part that you’ve alluded to. This is individualized and personalized, but it’s scaled at the same time. And I think there’s where we see a lot of the promise.
Eric Boyd: Exactly right. I think that’s very exciting that you can do this, and you can run it for a couple of hours and get lots and lots and lots of work done. And again, sort of with that theme of “this is AI that’s helping people do things better,” as an editor, you can review these and see how they’ve come in and really be able to go so much faster and so much more productive than you would in a different manner.
We see examples all over the place of how this AI is really helping people do things that they couldn’t before. We work with a lot of companies, and they come just in all shapes and sizes, and you really have to take them with the problems they have today and give them solutions that they can use today.
I think there’s already just such a democratization of technology that’s going on. When you think about how powerful your cellphone is versus who was able to get access to that type of computing power three decades ago, when you think about the access to the information on the internet and online versus three decades ago, we see that transformation happening just everywhere, where you’re putting power in the hands of so many more people. And so AI is both going to be a part of that and an accelerant of that, as I think about it.
When you think about the ability to create an application that right now requires knowing computer science and how to write code and knowing a programming language and having access to the tools to go and do that, and then you look at something like GitHub’s Copilot, which is just scratching the surface of how powerful it can be to describe a concept and have an AI literally translate that into code for you.
We’re going to have so many more people who, because of AI, are now able to create applications; they’re able to get work done that they previously couldn’t imagine getting done, that they might have needed to go hire someone to build something for them. I think we’re going to see a lot of that democratization continue to happen. Even with ChatGPT, I think we’re starting to see some of the democratization of “Let’s expose to the world, hey, this is the type of thing that’s possible.” It may be the ultimate homework cheater, and so we’ll have to deal with all the essays [that] now need to be filtered against “No, ChatGPT didn’t write this,” but just getting people exposed to the ideas of what you can do and then thinking about how that’s going to turn into the next companies, the next ways that are going to go in and power more and more people — that, to me, is where that democratization is really going to take off.
Sam Ransbotham: Pretty fascinating to think that Eric was talking to us right at the moment that this seemed to change the world.
Artificial intelligence became the kind of thing that a few people talked about to the kind of thing that everybody was talking about. Shervin and I, as part of our research with Boston Consulting Group and MIT Sloan Management Review, just released a report that finds that the use of artificial intelligence has jumped 20 [percentage points] since the last time we talked. It was 50% before, [up] to 70% now, and it’s hard not to think that generative AI is a big part of that. But this pervasive use comes with its own set of concerns, perhaps. Last season, in May of this year, we spoke with Mario Rodriguez from GitHub. He explained Copilot, another generative AI tool that’s broadly being used now, and how it got started.
We’re seeing that these tools can benefit not just individual uses, but they’re getting embedded in products, and those products are showing up everywhere. Let’s hear what Mario had to say.
Mario Rodriguez, senior vice president of product, GitHub: Product management — maybe this is a little bit controversial out there, but it’s a very new discipline, in my opinion. It’s not something super big. Like, we know how to do agriculture, as an example. Developers have been doing stuff for a very long time, too, one way or another. But product management as a discipline, I think, is probably new. I think Product’s role overall is to take something like an outcome that you want to achieve and then figure out how to get the product to be able to accomplish that outcome, and it’s magical to do that. People think it’s a set of features, and you then get into this thing called feature factories, if you have heard that term. And I think that’s not right.
The product discipline itself is more art than science, many times. You’re not there just to run stand-offs, or you’re not there just to talk to customers, either. There’s this beauty of the product discipline that is all about taking “What does a customer want to accomplish?” or “What does the business want to accomplish?” — both of them are super valid, right? — and then figuring out how to get there.
At least from my end, I love to do it incrementally. It’s very hard to be right all the time, and for me, what I always tell the product team is, how can we actually figure out what is the next incremental thing that we could do to validate a set of assumptions to try to get us there?
And that incrementality doesn’t have to show up to the customer right away. We do this thing at GitHub called staff ship — so we ship it first to ourselves. But that is a point where we could validate the idea, and that is a point where we could then figure out “Did this work or didn’t it, and what’s the next graph of the idea to accomplish that outcome?”
I usually say a product team is really there to take these ideas, marry them with outcomes, and they fill the middle of that equation.
GitHub Copilot is our AI assistant, and we’re kind of a little bit proud about this. We were the first copilot out there. And if you were internally within GitHub at the time that we started this — there was a big memo at one moment done by a set of researchers at GitHub. We have this division called GitHub Next, and they oversee the Horizon 2/Horizon 3 set of investments for GitHub. And we had just started to play with some of the models — some of these new LLM models coming from OpenAI. And the paper was about that: “Can we create this AI coding assistant that can really help you and supercharge you overall?”
So GitHub Copilot is the brand name for that, and we’ve been very successful with — again, going back into that iteration — iterating toward amazing value to developers. And, you know, we have stats like 55% improvement in productivity. And this one, I’m really proud of: We try to ask a lot of our customers to really measure developer happiness overall. We know that if developers are unhappy in an organization, not a lot of things are happening, right? You want to keep them happy. Talent out there is scarce, so you definitely want to keep your developers happy. We’ve been very successful in both the adoption side and the value side, but even more importantly, at least for me at times, is that developer happiness and increasing that across the world as well.
Shervin Khodabandeh: Mario talks about using GitHub tools internally, with staff ships to test, which also motivates and inspires employees when they see their products working. We’re seeing AI tools democratized and being used by so many people at work and for personal use.
Another thing we’ve seen is around sharing the knowledge we’re gaining and the mistakes we’re making. Last episode, I referenced a YouTube channel I found where employees literally show the mistakes they made in front of customers and how they fixed them to get happier customers in the end. How is the tech community doing this, especially in the wake of this insanely fast-moving world of GenAI? Our recent discussion with Rebecca Finlay of the Partnership on AI talks about openness and sharing.
Rebecca Finlay, CEO, Partnership on AI: We know that in order to be innovative, in order to be opening up new markets, in order to be thinking about new beneficial outcomes, you need to be thinking about how you are doing this safely and responsibly as well. The whole opportunity for generative AI to become much better — because today, it still has real challenges, whether it’s hallucinations or other ways in which it is deployed — just hasn’t really gotten to where it needs to be. But if we’re starting to think about, once it’s there, how it can be deployed to really deal with some of the biggest global challenges of our time.
That’s why I’m at the Partnership on AI: because I believe that AI does have that transformative potential to really support important breakthroughs, whether it’s in health care or, really, the big questions in front of us around our environment and about sustainability. We’re already seeing this in the predictive AI world, where we’re starting to see it just becoming integrated into the scientific process across all sorts of disciplines.
I do think, getting back to this question of the trade-off between responsibility and innovation, that one of the things that I hear from companies right now is, they feel alone as they’re trying to disentangle the risks of deploying these technologies and the benefits to their productivity and the innovation and how they serve their customers as well. And so one of the reasons why I think the work that we do at PAI is important is, I want to say there is a community of organizations that are wrestling with exactly these same questions, that are trying in real time to figure out, what does it mean to deploy this responsibly in your workforce? What does it mean to think about the safety of these systems and how they’re operating, whether that’s auditing oversight or disclosure or otherwise? How do you experiment, and what is best practice? And so I think more and more, if we can let companies and organizations know that there are communities who are actively working on these questions, where you can get some insights and, really, in real time, develop what will become best practice, that that’s a good thing for them to know.
Sam Ransbotham: That Partnership on AI is focused on community and knowledge-sharing aspects seems really big. There is a wave of trend here that is going on that shifted from AI products being supersecret to maintain competitive advantage and then maybe a shift toward openness, but that’s not entirely true — because some organizations are open and then closing again. We’re looking at OpenAI when we say that. Can we find a balance between competitiveness and thinking about how we all can learn together?
There are many, many benefits to openness — for example, open algorithms. Many, many models are available online, free for everyone to use, [such as] MapReduce, the Hadoop distributed file system. These algorithms came out of companies [that] shared them.
Data is important, too. We’ve seen that open data access enabled computer vision progress with ImageNet and the surge of deep learning that happened in 2012. Shervin and I had an episode with [the cofounder of] Nightingale Open Science, [which was] working to improve open-data-access in the health care context.
How can we foster all this? We talked with Joelle Pineau at Meta about how they’re using the algorithms and developing algorithms but then also sharing them with other people to benefit from.
Joelle Pineau, vice president of AI research, Meta: There’s a strong culture toward open protocols at Meta that predates the AI team. The basic stack — the basic software stack — is also based on many open protocols. And so that culture is there to this day; that culture continues. It goes all the way to the top of the leadership, and that commitment to open-sourcing the models is strongly supported by [Meta CEO] Mark Zuckerberg and his leadership team, so I don’t see that this is going to stop very soon.
What is going to be important is that we continue to release models in a way that is safe, and that’s a broader conversation than just one company. The governments have several points of view of how we should think about mitigating risks for this model. There’s also a lot of discussions about how to deal, in particular, with very frontier models — the largest, most capable models. And so we’re going to have to have these conversations as a society, beyond just the labs themselves.
I think one of the challenges is determining what we want out of these models, right? We’ve seen some pretty egregious examples recently of groups from [what] I assume is well-meaning intent to rebalance data sets, especially with representation of, for example, different racial groups in images. Of course, if someone asks for an image of an engineer, you don’t want only men to show up. You would hope to have a few women show up. And there are ways to rebalance the data, there are ways to recompensate at the algorithmic level. But sometimes you end up with very unusual results. And so it’s also a question of, what [is] the distribution of results that we expect and that we tolerate as a society? And in some cases, that’s not very well defined, especially when the representation is biased within the real world as well.
In many ways, deciding as a society what we want these models to optimize for and how we want to use them is a very complicated question. That’s also the reason why, at Meta, we often open-source the research models. We don’t necessarily open-source the models that are running into production. That would open us up, I think, to undue attacks, and it’s something we have to be careful about, but we often open our research models. And so that means, very early on, if there are major opportunities to improve them, we learn much faster, and so that gives us a way to, essentially, make sure that by the time a model makes it into product, it’s actually much better than the very first version. And we will release multiple versions as the research evolves, as we’ve seen, for example, with the Llama language models.
You know, we released Llama 1, Llama 2, Llama 3, and so on, and every generation gets significantly better. Some of that is, of course, the work of our own fabulous research teams, but some of that is also the contributions from the broader community, and these contributions come in different forms. You know, there are people who have better ways of mitigating, for example, safety risk, there are people who bring new data sets that are allowing us to evaluate new capabilities, and there are actually some very nice optimization tricks that allow us to train the models faster, and so all of that sort of converges to help make the models better over time.
Shervin Khodabandeh: To close out here, it’s hard not to spot a really cool example. Last December, we spoke with Vandi Verma, roboticist at NASA. She shares a really specific example of how NASA uses AI to simulate conditions on Mars.
I think a key point here is the Rover has some parameters on what to do and what to navigate but also [has] a lot of degrees of freedom to explore and to find out what’s going on. Vandi talks about, you know, we’ve never been to Mars, so we actually don’t know what we’re looking for. They allow the Rover quite a lot of degrees of freedom to explore and to stop and take pictures of whatever it finds interesting; no one ever defined for it what interesting is.
Vandi Verma, chief engineer for robotic operations at NASA’s Jet Propulsion Laboratory: I am the deputy manager for the Mobility & Robotics System at NASA’s Jet Propulsion Laboratory, and I’m also [working] with the chief engineer for the Mars 2020 mission, which consists of the Perseverance Rover and the Ingenuity helicopter.
JPL is a NASA center that specializes in building robots for space exploration. And NASA’s mission is to explore, discover, and expand knowledge for the benefit of humanity, and what we do is the robotics aspect of that.
AI has sort of transformed what we call that over a period of time, and there are things that we do on the ground, and there are things that we do onboard our robots, and so I’m going to touch on some of those. So, in general, we are more on the side of autonomous capability — closer to what you might think of as what self-driving cars use — and not a lot of it is potentially classically machine learning per se, although we use that to inform a lot of our work.
In fact, with Perseverance, 88% of the driving that we’ve done is autonomous driving. And so the Rover has cameras: It’s taking the images; it’s detecting the terrain and figuring out what’s hazardous and navigating around obstacles. And it’s actually quite interesting because it’s driving on terrain that no human has ever seen, so we can’t even give it that kind of information. So that is definitely a form of autonomous navigation.
We also, at the end of drives, are trying to make a lot of progress because we’re in this really harsh environment, and we have a mission to collect and cache a certain number of samples with Perseverance, because for the first time, we are actually going to bring them back to Earth. But we want them to be from as distinct places as possible, so we want to do a lot of driving. If you stop all the time, you’re not going to make as much progress. But who knows if there’s something really exciting along the way that we’re just going to miss? In our world, we call it the dinosaur bones.
We have AI capabilities on the Rover where it’ll take a wide-angle image, look at a large swath of terrain, and then try to figure out what is the most interesting feature in there. We have a whole slew of instruments, but one of the instruments is the SuperCam instrument, which does a lot. It has a laser, and from a distance, you can shoot a laser at a rock, and it creates a plasma, and we study that with a telescopic lens. That is such a narrow field of view — you know, a milliradian — and so if you were to try and do that to the whole view you see, you’d spend days there.
So essentially, we use the AI to figure out “What’s the most interesting thing that we should zap?” And then you can send the data back and tell the scientists on Earth. That’s been very valuable as well.
I think one of the most interesting things about defining what’s interesting is that it puts it on the humans. We actually have a really hard time telling machines what we want them to do, right? In order for us to tell what’s interesting, we have a lot of different parameters that scientists can use to specify “I am looking for light-toned rocks of a particular size, of a particular albedo and shape, that are interesting in this area.” And we can change that. So we have these different templates, depending on the terrain we are in, that scientists on the ground help us determine. We send that to the robot to say, “We’re looking for this kind of thing.”
We have done some research as well, where we tell it, “You now track all of the things we have seen” — it’s called novelty detection, which we don’t actually yet have deployed, but “Find what we haven’t already looked at.” That’s another one.
But there are two things in here. When we’re doing exploration, we’re looking for things that are new, but we also try to characterize things we have seen with multiple different instruments, because we are trying to collect a statistically significant amount of data for the hypothesis we have. We’re trying to figure out “Could life have existed on Mars and, especially, ancient life?”
And so that puzzle. … There are hypotheses, and you’re trying to answer specific questions, and that’s what the scientists then will tell the robot that they’re interested in. We’ve actually used supercomputers to translate that into parameters that we can then uplink to the robot.
This is, I think, an area where AI can help a lot because we are still in that phase in robotics in a lot of areas where we have a lot of knobs. We can do a lot, but the art is in tuning this multivariable space. In fact, you know, just on Perseverance — we call them parameters in software, [and] this isn’t even taking into account hardware design and other things — we have over 64,000 explicit parameters. These are saved in nonvolatile memory. This is not even taking into account the arguments to commands you can send. So there’s just so many ways in which you can express what you have to say, and that’s where we can use a lot of capability to know what the right combination is for what we intended to do.
Sam Ransbotham: I really liked the NASA episode. It was one of my favorites. But that’s like asking which of your children is your favorite. If my children are listening, both of my children are my favorite.
NASA is still exploring what we’re all exploring. Exactly zero people in the world have 10 years’ experience with generative AI, no matter what the job postings ask for. We’re all learning as we go. We’re going to increasingly use generative AI and AI tools in places that are not obvious and are not clear at first. That’s OK. We all are learning together about these things.
There is a lot of uncertainty about the things we are learning. Shervin and I have a report that came out about this exact relationship between organizational learning and uncertainty. We’ll link to that in the show notes, as well as to each of these full episodes.
We’ve had inspiring conversations with leaders in supply chain, marketing, software development, but if there are other topics that you’d like us to [take] a deep dive on, please comment in your reviews or send us an email. Again, we’re all collectively learning how to use these tools; they never existed before. And we’ll have more guests coming along that we can learn from.
Thanks for listening. We’re back soon with a conversation with Ronald den Elzen, chief digital and technology officer at Heineken. Please join us.
Allison Ryder: Thanks for listening to Me, Myself, and AI. Our show is able to continue, in large part, due to listener support. Your streams and downloads make a big difference. If you have a moment, please consider leaving us an Apple podcasts review or a rating on Spotify. And share our show with others you think might find it interesting and helpful.
