Topics
Artificial Intelligence and Business Strategy
In collaboration with
BCG
In collaboration with
When Mario Rodriguez emigrated from Cuba to the United States at age 14 with his parents — a university professor, and a teacher turned electrical engineer — they had already instilled in him the value of education and a love of learning. That passion has guided him throughout his career — as a program manager with Microsoft; then as part of GitHub, following Microsoft’s 2018 acquisition of the developer platform; and as a cofounder of a charter school in North Carolina. Now, as senior vice president of product at GitHub, Mario oversees the team developing the GitHub Copilot AI-assisted software development tool.
Mario joins this episode of the Me, Myself, and AI podcast to share his views on product development, GitHub Copilot’s effect on productivity and job satisfaction, and a future in which more people can easily develop their own software.
Subscribe to Me, Myself, and AI on Apple Podcasts or Spotify.
Transcript
Sam Ransbotham: Stable products typically require extensive testing to make sure they’re reliable for customers. But products based on inherently changing tools like generative AI fundamentally change that. Find out how on today’s episode.
Mario Rodriguez: I’m Mario Rodriguez from GitHub, and you’re listening to Me, Myself, and AI.
Sam Ransbotham: Welcome to Me, Myself, and AI, a podcast on artificial intelligence in business. Each episode, we introduce you to someone innovating with AI. I’m Sam Ransbotham, professor of analytics at Boston College. I’m also the AI and business strategy guest editor at MIT Sloan Management Review.
Shervin Khodabandeh: And I’m Shervin Khodabandeh, senior partner with BCG and one of the leaders of our AI business. Together, MIT SMR and BCG have been researching and publishing on AI since 2017, interviewing hundreds of practitioners and surveying thousands of companies on what it takes to build and to deploy and scale AI capabilities and really transform the way organizations operate.
Hello everyone. Today, Sam and I are excited to be speaking with Mario Rodriguez, senior vice president of product at GitHub. Mario, welcome to the show.
Mario Rodriguez: Hello. How are we doing?
Shervin Khodabandeh: We’re doing great.
Sam Ransbotham: I think most of our listeners know what GitHub is, but can you describe it briefly for us?
Mario Rodriguez: Yeah, absolutely. So we usually describe GitHub as the home of all developers. It is the place where you store your code and other assets that are necessary to develop and ship software. It’s also a place where you come to collaborate and remix ideas with the open-source community. I have two daughters. One of them is 6 years old, and sometimes the way that I describe what daddy does is, my daughter has a book bag and she puts a bunch of things necessary in there — you know, her lunch, her snack, and her folder; she has a purple folder where all the homework goes and things like that. And that is the space for her to accomplish the stuff that she needs to do at school.
And I think about GitHub being that for developers. It’s the place where you come and put everything in order to develop and ship software. And we’re lucky to have an amazing community on top of it.
And my role is, I am the SVP of product at GitHub, and I oversee both our product and design teams, including GitHub Copilot. I joined GitHub as part of the Microsoft acquisition in 2018. In fact, I’ve been working in dev tools for 20-plus years. I consider myself super lucky in the sense that I love what I do for a living. I sometimes have to pinch myself on that.
I think about it … humanity already created the wheel, created fire. Everything that we’re going to be doing going forward to advance it is going to be through software. And giving the software developers amazing tools to continue to push humanity forward is a lucky place to be, and I just love that mission and that drive. So I consider myself lucky in that. Right now, I’m overseeing our product and design teams and making sure that we continue to innovate and we continue to meet the needs of our customers.
Shervin Khodabandeh: I couldn’t agree more with what you said about the importance of enabling developers to collaborate more, particularly as everything is moving more and more toward that. Tell us more about what the product group does.
Mario Rodriguez: Product management — maybe this is a little bit controversial out there, but it’s a very new discipline, in my opinion. It’s not something super big. Like, we know how to do agriculture, as an example. Developers have been doing stuff for a very long time, too, in one way or another. But product management as a discipline, I think, is probably new. I think Product’s role overall is to take something like an outcome that you want to achieve and then figure out how to get the product to be able to accomplish that outcome, and it’s magical to do that. People think it’s a set of features, and you then get into this thing called feature factories, if you have heard that term. And I think that’s not right.
The product discipline itself is more art than science, many times. You’re not there just to run stand-offs, or you’re not there just to talk to customers, either. There’s this beauty of the product discipline that is all about taking “What does a customer want to accomplish?” or “What does the business want to accomplish?” — both of them are super valid, right? — and then figuring out how to get there.
At least from my end, I love to do it incrementally. It’s very hard to be right all the time, and for me, what I always tell the product team is, how can we actually figure out what is the next incremental thing that we could do to validate a set of assumptions to try to get us there?
And that incrementality doesn’t have to show up to the customer right away. We do this thing at GitHub called staff ship — so we ship it first to ourselves. But that is a point where we could validate the idea, and that is a point where we could then figure out “Did this work or didn’t it, and what’s the next graph of the idea to accomplish that outcome?”
I usually say a product team is really there to take these ideas, marry them with outcomes, and they fill the middle of that equation.
Shervin Khodabandeh: Mm-hmm. And they do it iteratively — not just [getting] it right the first time, right? Which is, I think, what you were talking about as the mental shift in, like, it’s not a big bang. And the more we get comfortable … some of the greatest products have been introduced and evolved over many, many iterations.
Mario Rodriguez: One hundred percent. And you build on top of each other, too. Like, you might try this … By the way, the same thing, in my opinion, happens in medicine and in other industries as well. They do it very well. The biggest breakthroughs have happened sometimes by chance by just testing a bunch of things, right? Thomas Edison and the light bulb, right? It took a lot of iterations to get there. And you would think we would have learned by now that the way to accomplish the biggest breakthroughs is just through that iteration, but you do get, at times, a set of beliefs that product management is all about managing a project and things like that. And the reality is, it’s just about iteration and finding the next set of things that will accomplish the outcome that you want, 100%.
Sam Ransbotham: So in that middle ground … you mentioned Copilot earlier. Where does Copilot fit in that middle ground? Actually, tell people what Copilot is, to start with, and, then, how does that fit in the middle ground?
Mario Rodriguez: GitHub Copilot is our AI assistant, and we’re kind of a little bit proud about this. We were the first copilot out there. And if you were internally within GitHub at the time that we started this — so it was a big memo at one moment done by a set of researchers at GitHub. We have this division called GitHub Next, and they oversee the Horizon 2/Horizon 3 set of investments for GitHub. And we had just started to play with some of the models — some of these new LLM models coming from OpenAI. And the paper was about that: Can we create this AI coding assistant that can really help you and supercharge you overall? So GitHub Copilot is the brand name for that, and we’ve been very successful with — again, going back into that iteration — iterating toward amazing value to developers.
And, you know, we have stats like 55% improvement in productivity. And this one, I’m really proud of: We try to ask a lot of our customers to really measure developer happiness overall. We know that if developers are unhappy in an organization, not a lot of things are happening, right? You want to keep them happy. Talent out there is scarce, so you definitely want to keep your developers happy. We’ve been very successful in both the adoption side and the value side, but even more importantly, at least for me at times, is that developer happiness and increasing that across the world as well.
Shervin Khodabandeh: I wanted to ask you about that. We’ve seen a variety of folks get far more effective and efficient in writing and debugging and even thinking through how to structure code and often things that won’t even come across a human mind because of how we have maybe this herd thinking or whatever, and sometimes code is much simpler and much cleaner when you are in the Copilot mode. Yet what I’ve seen and heard, somewhat anecdotally, is also, folks are adopting it and they’re loving it, and then there are folks who just don’t even go near it and they’re like, “It’s not for us.” Why do you think that is? Is it fear of “I’m going to lose my job and this thing is going to take over”? Why do you think there is maybe a little bit of a bifurcation of reactions to this?
Mario Rodriguez: Yeah. I love that question, Shervin. Maybe let’s try to explore [it from] three different angles. No. 1, going back to something that Sam had asked: How do we think about Copilot and the value that Copilot does? And how do we even do the product GitHub Copilot? I would say AI products, and developing these AI products, is completely different than any other products or features that we do.
That has been one of the biggest shifts, I think me, my team, and even GitHub has gone through, which is, if you think about just a normal feature that we do, we understand, “OK, this is kind of the outcome that we want out of the feature. Here’s some code. Well, we could write unit tests for that code. We could test it.”
We kind of found it’s very deterministic. We know if something is working or not. We know if there’s a bug or not. We know how to fix, many times, that bug. We could see even the error that we’re making while developing the feature. LLMs are not deterministic. They’re not. And you cannot just expect them to be deterministic. So how do you end up providing value within a nondeterministic set of inputs that are at your disposal there? So one of the biggest things that I would say to the community at large, if you’re going to start developing features against … I would say AI features or features against LLMs is, you have to rethink how you’re thinking about quality. You have to rethink how you’re approaching the feature.
We have all of these evaluations and offline evaluations that we do for GitHub Copilot. And that’s how we determine the quality of what it is giving you in something like code completion. So going back to the adoption side, we know that many times, Copilot is going to be generating around 30%, 40%, 50-plus percent of your code. The acceptance that you have, depending on how many lines and stuff like that, is in the 30-plus percent for many of these developers.
And we could see it in our offline evaluation as well, where Copilot does really, really well and where Copilot needs to continue to improve — meaning we probably need to pass it a lot more context for it to generate the right piece of code for you because code and code development has a lot to do with the context that you’re dealing with. Are you in a legacy code base or not? Are you in COBOL or in C++ or in JavaScript or TypeScript? So it’s a lot of context that needs to happen for the quality of that code to be high and for you to accept it.
But we have this offline evaluation that we utilize for it. So a lot of the adoption today is coming because it’s providing you value. We strongly believe that the human has to be at the center at all times. But you have this Copilot that is very patient, right? It doesn’t get upset if you ask it “You know, how does it actually work to do X, Y, and Z within this code base?” or “I forgot how to write this register. Can you just please tell me what’s the best way of doing that?” Or …
Shervin Khodabandeh: “Tell me again, even though you told me eight times. Don’t judge me.”
Mario Rodriguez: Yeah. And it is incredibly patient and very empathetic.
Shervin Khodabandeh: Not judging, yeah.
Mario Rodriguez: And especially if you ask it with “please” and things like that, believe it or not, it does actually matter, and it does respond at times. So you’re like, “Did [I get a response] just by me asking, ‘Please’?” But it does actually respond very well to empathy for sure. So I do think it creates this productivity polarity out there, and that’s why the adoption is high. It’s because the people that use it end up being significantly more productive, and you see the difference. You see the difference in their output. You see the difference at times in the code quality, and then you see the difference as well in their happiness because they don’t have to write 10,000 lines of tests when Copilot can help them with that.
And then there’s this magical moment where you’re like, just through a comment — and, by the way, this is the thing that won me over back in the day — I think it was early 2021 when I first saw it. Because we had IntelliSense before, we had a bunch of other technologies that could complete code for you, by the way, but we never had the ability to start something with a comment to describe what the next code should be.
Sam Ransbotham: I wonder if the IntelliSense example has anchored people. If I think about IntelliSense, it was about filling in while you started typing in “for,” and it would open parentheses for you and do that sort of stuff. But the Copilot is a different level. And when I think about that, you mentioned a 30% code … 30% or 40%. I’m guessing it’s really not the interesting code. I think there’s probably a distinction here that that 30% …
Shervin Khodabandeh: Is that really the case, though? I think it could be quite interesting code.
Mario Rodriguez: It could be quite interesting code as well. Again, because of a comment that you can make, it could get you started. So you’re right; it’s not going to write the entire [thing] — although, with more of these agents that we’re developing, more and more of that interesting code can be developed. But just with a human at the center, whatever they describe, and you have heard this term “prompt engineering” and all of those things. Actually, prompting Copilot … the right way can help it generate very interesting code. And we see this; we have a Slack channel internally where people post what Copilot did for them at times, and you see some things in there that you’re like, “Wow, it did that? That is amazing.”
Sam Ransbotham: I guess I was thinking about, as we approach a task, the easy ones are going to be first, and the later ones will be harder.
Mario Rodriguez: One hundred percent — and then the latest one is going to be hard. Also, maybe let’s try to explore this angle or this question too. I would love to get your opinions on this. So imagine if the prompt to all of us is, what does the world look like where 10% of the population can code? Just think about that. Because sometimes I get out of bed asking myself this question: What needs to completely change for 10% of the population to be able to be called a developer?
Because today, you don’t have a lot of people throughout the entirety of the world with a computer, as an example — especially with advanced graphics and CPUs and all of those types of things. A lot of people have phones, but not a lot of people have computers in their house. And so I think that’s very interesting. Like, what needs to change? And for me, I believe the definition of a software developer will start changing going forward. And the interesting part of it is, probably all of us know how to cook, or all of us know how to bike, or all of us know how to swim, but, look, I am not opening a Michelin-star restaurant tomorrow.
But the rest of humanity — what if a text box is all they needed to be able to accomplish something that creates software and something that they could then derive value from? You [would] consider that, in my opinion, a software developer. If I could tell it very quickly in my phone, “Hey, I am thinking of talking to my daughter about these things. Can you give me the last three X, Y, and Z articles and then just create a little program that we could play as a game?” You could envision Copilot being able to help you with that in the future.
Sam Ransbotham: Yeah, I’d be curious what that 10% is now — what percentage is there now. But one thing that’s interesting, if I combine what you just said with GitHub in general, you said a lot about “Oh, I can make this thing do what I want it to do, highly personalized.” But then GitHub is also about sharing that, which I think is a very interesting dynamic that would play out. If you could both create something personally and then widely distribute it for others to improve and iterate on, that seems maybe even bigger than the 10% because you get the multiplicative of the 10% and then everyone else improving it.
Mario Rodriguez: One hundred percent. You know, the beauty of communities is that you get network effects on those communities, right? This is how languages work too. You get network effects because someone else knows your language and then you can communicate more broadly. So the beauty of community is that … and that is what unlocks, in my opinion, creativity as well, and it unlocks innovation. Imagine, then, if this little program that Copilot created for you, you could also share it and say, “Hey, this is what I created.” And you could remix it, and you keep remixing and remixing. Artists do this all the time, by the way. Picasso started not what you know today as cubism; it started completely different. It started actually in very realistic painting, very realistic pictures, and then kind of evolved from there and evolved from there until it found its own style, and that’s beautiful. So I think tools like Copilot are going to help you do that, and it’s going to be unbelievable what our children are going to experience through the advance of this technology.
And, look, we also need to get significantly better in developing AI tools as well. We’re in nascent mode right now.
Sam Ransbotham: One thing that I really liked is, you mentioned the idea of sparking someone’s interest. Maybe they see something and then that clicks for them, and then they decide to go deeper, but this increased exposure to lots of things gives us more opportunity for things to click. I don’t want my dean to listen to this, but I think that a lot of what we do in the classroom is not so much teaching but exposing to lots of ideas, and then some people … this will click with one person and this will click with another. I really liked your analogy there with coding — that maybe there’s some part of it that clicks and sparks some interest, or maybe there’s something that pushes someone to go forward. But I think it’s also true — it makes sense too — that not everybody is going to be that superstar or wants to be that superstar.
Shervin Khodabandeh: Mario, I wanted to ask you about your founding a charter school.
Mario Rodriguez: Oh, yes; we haven’t talked about that.
Shervin Khodabandeh: Tell us about that.
Mario Rodriguez: I was born in Cuba, and I lived there until I was around 14 years old. I came to the United States, and I didn’t speak a word of English. In fact, I went directly into high school. [I do] not highly recommend to go directly to high school without speaking the language, but we made it happen. And the reason I’m saying that is when I was in Cuba, my mother was a teacher at a university. And I spent a lot of time with her in kind of helping her grade tests and kind of just seeing what an educator does. And to tell you the truth, I was fascinated by it. My father also was a physics teacher at one moment, then he eventually became an electrical engineer. And when I got to the United States, I was like, “You know what? I’m probably going to study electrical engineering and also do something, maybe get a Ph.D., so eventually, if I want to go back to school, I could do that.” And all of a sudden, I found myself — you know, I was working at Microsoft and then eventually GitHub. And I was in North Carolina, and I noticed that in the rural parts and certain parts of North Carolina, it was really hard to attract great teachers there and maintain that and continually to push education forward, especially in things like computers or programming and things like that.
And we got together and we said, “What if we opened a STEM school with a slightly different take on it and we allowed the students to — you know, how Google, at times, I think has 10% time or 20% time or something like that? What if they actually, some of these students, can own their own curriculum at times, and they could learn at their own pace on this 20% time as well? And what if we don’t have so many walls between the classrooms, and we could sometimes unite classrooms together and have them really share?” — to me, talking about community, really share. So we opened a charter school just to explore those ideas and to be able to provide better education in places where it’s really hard to do that and sustain it at a high level.
So think about iterative correctness. I think about the school as a product, and you have to continuously try new things and try to see how it evolves. And we were so successful — we just did a middle school to start — we were so successful with that that we are opening a high school now. We did that last year.
I love education. I think it’s a great way to give back overall. And then the more we could do then, again, in technology and arming the students with both sides — the art side and the science side — then the better we’ll all be.
Shervin Khodabandeh: That’s wonderful. That’s a great story.
Sam Ransbotham: Let me transition here to … we have a segment where we ask you some short, rapid-fire questions, so just give us a quick answer here. What do you think the biggest opportunity is for AI right now?
Mario Rodriguez: I think the biggest opportunity is productivity right now, and, at least in software development, it would be democratizing software development overall.
Sam Ransbotham: Yeah, that’s interesting you mentioned the productivity versus the democratization, which are very different twists on that. What’s the biggest misconception people have right now with artificial intelligence? What are people getting wrong?
Mario Rodriguez: That it’s going to take jobs away, I think, is the biggest misconception, and that it’s going to actually then bring humanity backwards instead of forward.
Sam Ransbotham: Yeah. What’s the first career you wanted?
Mario Rodriguez: So my first career … I would say I was always an entrepreneur. So the first thing that I did to make money was actually selling things so you could do kites overall. So I sold these kite kits, and that was the first thing that I ended up doing.
Sam Ransbotham: So when is there too much AI?
Mario Rodriguez: When the human is not at the center.
Sam Ransbotham: Nice. What’s one thing you wish AI could do right now that it can’t do?
Mario Rodriguez: Transpile code in any language that I want and do that with high quality. If that could happen, that would be amazing because then I could put something that I wrote in TypeScript and try to see “OK, how would that then …” — or in Python, as an example: “How would that then look like in these other languages?” I always find fascinating different programming languages — what’s good about them or not. You know, Lisp, as an example — is very hard to read at times because of all the open brackets. But that’s what I would say.
Sam Ransbotham: That’s interesting. So I think one of the things we covered today is just the wide gamut of possibilities here. One of the things you’ve kind of opened my eyes to is this idea of what happens when even the percentage of people who can code in the world changes — what could that do? I think that’s really interesting for us to think about, both positive and negative; that has some big implications. Thanks for taking the time to talk with us today. We’ve enjoyed it.
Mario Rodriguez: Thank you all. Pleasure to be here.
Shervin Khodabandeh: Thanks for tuning in today. Next time, Sam and I meet Patrick Hebron. Please join us.
Allison Ryder: Thanks for listening to Me, Myself, and AI. We believe, like you, that the conversation about AI implementation doesn’t start and stop with this podcast. That’s why we’ve created a group on LinkedIn specifically for listeners like you. It’s called AI for Leaders, and if you join us, you can chat with show creators and hosts, ask your own questions, share your insights, and gain access to valuable resources about AI implementation from MIT SMR and BCG. You can access it by visiting mitsmr.com/AIforLeaders. We’ll put that link in the show notes, and we hope to see you there.
