The Quest to Measure Basketball IQ

Basketball IQ is one of the game’s holy analytical grails. We know it’s there, or at least we think we know it’s there. But what is it even made of? Intuition? Vision? Knowledge of the game? Or is it simply natural intelligence and brain processing speed applied to a specific context?

Former director of sports analytics at ESPN Ben Alamar is one of sports’ leading proponents for and experts in basketball IQ. In this episode, Ben joins the show to defend the hypothesis: NBA teams would make fewer draft mistakes if they measured basketball IQ.

Transcript

Paul Michelman: In 2003, a new question was added to the annual survey of the NBA’s general managers: “Which player has the best court smarts?” First place went to New Jersey Nets’ superstar Jason Kidd — a ballyhooed prospect out of high school, an All-American at Cal, and by this time, a six-time All-Star in the NBA. Second place went to Tim Duncan of the San Antonio Spurs, who put even Kidd’s resume to shame — two-time All-American at Wake Forest, two-time NBA Champion, and two-time league MVP. But the third-place finisher, unlike Kidd and Duncan, wasn’t an obvious future first-ballot Hall-of-Famer. Steve Nash had grown up in Canada, not exactly a basketball-crazed country. He had attended Santa Clara University, known more for its contributions to Silicon Valley than to basketball. And it had taken him until his fifth season in the NBA to average more than 10 points per game, which barely clears the bar of average. But Nash had carved out an important leadership role as a Dallas Maverick, and in the eyes of the league’s GMs, had shown how a player lacking in classical athleticism but versed in “court smarts” could find success at the highest levels of the game. Here he is speaking in the ESPN film Basketball: A Love Story. Steve Nash: “I was always waiting for the defense to make a mistake. My last resort was to shoot. In a way I was like an architect trying to solve a puzzle about space. That was really the way I saw the game.”

Ben Shields: Basketball has always shone a spotlight on the impressive physical properties of its players. Who can’t help but marvel at Mo Bamba’s 7-foot-10 wingspan, Zach LaVine’s 46-inch vertical, or John Wall flying to the basket in a supersonic blur? But there is a number that you won’t be able to find on any scouting report or stat leaderboard — a player’s basketball IQ — and it might be the most important measurable of all. Or else how would Steve Nash, the 6-foot-3 pride of Victoria, British Columbia, go on to win two MVP awards, rack up assist titles, and enter the Basketball Hall of Fame this past summer as one of the greatest point guards who ever lived. I’m Ben Shields.

Paul Michelman: I’m Paul Michelman. And you’re listening to Counterpoints, the sports analytics podcast from MIT Sloan Management Review.

Ben Alamar: In this episode, the curious case of Steve Nash. And the elusive quest to measure basketball intelligence.

Ben Shields: In each episode of Counterpoints, we look beyond the data in search of what the data reveals…

Paul Michelman: (interjecting): Or supposedly reveals.

Ben Shields: …about what’s actually happening both on the field and off.

Paul Michelman: We put one analytics-based hypothesis to the test and see how well it stands up.

Ben Shields: Today’s hypothesis: NBA teams would make fewer draft mistakes if they measured basketball IQ.

Paul Michelman: Basketball IQ is one of the game’s holy analytical grails. We know it’s there, or at least we think we know it’s there. But we’ve never been able to put a gauge on it. And are we even sure what it’s made of? Intuition? Vision? Knowledge of the game? Or is it simply natural intelligence and brain processing speed applied to a specific context? And that makes any hypothesis about basketball IQ especially challenging.

Ben Shields: Which brings us to Ben Alamar, former director of sports analytics at ESPN who is now a consultant working in the field. He is one of sports’ leading proponents for and experts in basketball IQ. We asked Ben to defend the thesis, and we started with a definition.

Ben Alamar: The broad concepts that I use — you can talk about intelligence or basketball reflex or a variety of different words — but the general concept is that the very best players in the world can understand what’s going to happen on the court before everybody else does. So they’re not reacting to what’s happening right now. They’re reacting to what’s going to be happening in two or three, four, five seconds. It’s like a chess master who is thinking four, five, six moves ahead. Or as Wayne Gretzky once said, you don’t skate to where the puck is. You skate to where the puck is going to be. And that’s what I try to talk about and try and measure what I’m talking about — the basketball intelligence.

Ben Shields: How do you measure basketball intelligence? I like the fact that you’re starting to think about, well, how is that player going to be five, six, seven plays ahead of the competition. But, how are you actually thinking about measuring that intelligence?

Ben Alamar: You can think about a variety of different ways, and most of the instruments to try and do it right now are actually pretty blunt. We are not going to get a precise measurement under most circumstances. However, we can start to measure, and even if we can get a little bit of an understanding, a little bit of clarity into this question, that can help us make better decisions. So, the idea is this, you’re looking at a lot of research, academic research around experts. We know that experts in their field make decisions faster and more accurately than anybody else.

So, to take it right into sports, if you took a group of professional basketball players and have them make basketball decisions about how something is happening on the court, they should be able to make those decisions faster and more accurately than coaches, than fans, than anybody else. And so, one way to get at this idea of basketball intelligence is to show the athletes some film of a play developing, and basically say what’s happening next? And there are a variety of different scenarios you can present to them in that way.

So, for one example, show them a fast break as it is developing, and ask them who is going to score on the play? They watch two or three seconds of the fast break developing, the video stops, and they have to choose which player is going to be scoring on that play. They are basically using the information they have from watching the film and understanding the game to see what’s not from a basketball sense the schematically “right” answer, but what’s actually going to happen, and so the ones who can answer that kind of question or other similar kinds of questions faster and more accurately should have higher basketball intelligence, and therefore be more likely to be elite-level basketball players.

Ben Shields: These two metrics of faster and more accurate. Let’s talk about some players. Let’s get into this discussion and how would you apply these metrics to a specific example of a player.

Ben Alamar: I think really, it starts with why I started to go down this path in the first place. And that is, I was doing draft analytics work, using college performance to project NBA success, and you build these models, and you know, I worked with the Trailblazers and the Thunder for many years doing these kinds of models. And every time I built one, I kept noticing that one player, I couldn’t — the models got better every year, but Steve Nash always failed to register as one of the best players. Like, I couldn’t build the model without building just the Steve Nash Model, which is not good statistics without Steve Nash being in sort of the top tier, and that seemed wrong and it bothered me, and I started to refer to Steve Nash as my nemesis. Like, I got to figure out this guy, right?

So that’s where I start to go down this path of like, all right, so what are we missing? What is the information that is missing that we need? And so the concept then becomes, all right, so college players are, you know, you can do really well in college by being a great athlete. If you’re a great athlete, you can perform well in the college game. But everybody in the NBA is, you know, the minimum standard of athleticism in the NBA is higher than college. And so you can’t just bull your way through with just that pure athleticism in the NBA. There are too many examples we’ve seen. So in thinking about Steve Nash and thinking about the draft in general, it’s like we make mistakes based on athleticism, because most of our information is either a pure athleticism — they combine measurements, speed, height, all that kind of stuff, you know vertical jump — or it’s colored by athleticism in terms of college performance.

The piece that’s missing is this basketball intelligence. We don’t have a pure measure of it. So I designed this tool that would provide the video, collect the answers, time the answers, create how accurate the answers were. And when I was with the Thunder, we took this tool to the Portsmouth Invitational Tournament, which is a collection of college players who are hoping to, if they’re really, really lucky, be the last pick in the NBA draft but are probably going to be undrafted free agents. And most of the people at the Portsmouth Invitational are never going to play in the NBA.

And so the Thunder took this tool and they gave it to around 20 athletes there. It was a brief, five-minute quiz. I think there were about 15 questions that they answered, and what was really interesting about that data is that everybody did pretty badly, except for one player. One player scored well both on speed and accuracy, like well above everybody and in the neighborhood of where we would expect a professional player to be relative to everybody else we experimented with this on.

He was the only player, none of those players ended up getting drafted, including this guy. But he’s the only one to ever have a career in the NBA. It’s Kent Basemore, who had a, he’s having a reasonable NBA career. My thesis then, obviously we don’t have the data to prove this in any real way just yet, but the thesis is, he separates himself because he has basketball intelligence and his data, much like Steve Nash’s data, probably didn’t reveal that in college in the same way that it can be an advantage in the NBA.

Ben Shields: Yeah, that example of Nash is obviously a good one, and Basemore is a great one as well, because he has exceeded a lot of expectations and in a lot of ways came out of nowhere. And what we’re hearing is that, you know, the Thunder had better information, so to speak, about him as a player and ultimately that led to a shot that he got with the NBA, and he is making the most of it as well.

Ben Alamar: Yeah. I want to be careful not to oversell what we did because it was really an experiment. We had this data point basically on him. We did not have a large enough data set to really make a decision about it, but it is a strong anecdotal piece of evidence that this kind of information, particularly as we get better and better at measuring it, is going to be really important in making these kinds of decisions.

Ben Shields: I’m glad you brought that up, because one of the goals of this podcast is we want to kind of get the additional data that is required for answering these types of questions. So I think, thinking creatively, thinking big picture here, innovatively, what are some of the data sources that you would love to have to more effectively measure basketball IQ?

Ben Alamar: Well, the easiest thing from a data perspective, to really get the best measurements, we need to hook everybody up to brain scan machines. Because what we do know about experts is they actually process the information from their field of expertise in a different part of their brain than everybody else. And so if we could show basketball, you know, hook up a player to brain scans and have them watch basketball and watch where their brain is reacting and processing things, then we’d know whether they have a well-developed basketball intelligence or not. That’s probably not logistically possible. But we do have some other things. We have this sort of blunt tool that I’ve developed, and if we can develop that more — get it more precise, more scenarios, more situations — it becomes a longer test, and something that you really have to take time to put an athlete through, which is really one of the pushbacks from the front offices.

They don’t have a lot of time to work, to spend with these draftees, and so they don’t want to use 20 minutes in this test that at this point is not really proven. But now there have been developments in sports data and that I think can really help and give us an edge to do this. And that is what they call player tracking data. We can use cameras to optically track everything that moves on a basketball court 25 times a second. So now we know where everybody’s moving, constantly, all the time. With that kind of data, you can devise systems to measure what people are reacting to and when people are reacting to them.

So for example, you take an athlete, a defender, and watch when a player takes a shot, when the opposing team takes a shot. How are they reacting to that, and are they reacting differently whether the shot eventually goes in or is missed? Then we know whether that player is making the prediction in their own head, reacting to a missed shot or a made shot differently, and making accurate predictions about it.

And so those are the kinds of situations we can start to measure and find these types of basketball intelligence measures that, eventually, we can build up into a real, all-encompassing measure of basketball intelligence that matters.

Ben Shields: Yeah, I like that notion of basketball intelligence that matters. Certainly when you look at the future here, and you presented a bunch of compelling ideas, whether it be brain scans in the future or leveraging the player tracking data today. I think just a few questions about limitations here that I think are important to talk about. And I want to get your reaction to some of these questions. So, is basketball IQ a by-product of the coach’s system? Or, do you see it as specific to the player’s basketball expertise? We get into this debate a lot in football for instance, about the success of a quarterback. What is the variable of the offensive or defensive system as it relates to basketball IQ?

Ben Alamar: Yeah. I think basketball intelligence doesn’t manifest itself as one thing. It’s different in each player. What we care most about is whether it’s at a high level. If it’s at a high level, then a good coach should be able to put that player in a position to succeed. Some systems though, there will be some times where a player is just a bad fit to a system. So no matter how smart they are, it’s just a bad fit. I mean, we can imagine if you take somebody like Lebron, and he is told to play the five all the time and just do what a traditional five does. That would not be a good situation. He would probably do fine, but he wouldn’t be Lebron. And so, the kind of system is important and affects performance, but to succeed at all in any system, at the NBA level, at the truly elite level, there’s got to be this level of intelligence.

Ben Shields: All right, good. I’m following you there. Let me pick up on this level of intelligence and ask you this question: Can someone have a high basketball IQ without a high level of natural intelligence?

Ben Alamar: Sure.

Ben Shields: And if so, how are teams, if at all, measuring intelligence generally?

Ben Alamar: So, I can speak only to the teams that I’ve worked for, in terms of the general testing. And I’ve seen a variety of things in the marketplace that people are trying to sell the teams. In general, not necessarily, there’s some intelligence testing — you know, the NFL has their measure that they do during the combine. They’re all very general in nature, and some of them are general psychological testing about profiles and things. None of those things are really validated against performance on the court or the field of play in any way that I’ve seen. You can have somebody who’s not very bright and still be a great, great basketball player. Typically, the very, very best are usually pretty intelligent people, but there’s no requirement that you have to be a genius in general to be a superior athlete. You just have to put the time in what is often referred to in the literature as “deliberate practice” to get to a point where your brain is functioning and processing things in a way an elite basketball player does.

Ben Shields: That’s great. All right, we’re going to get you out of here on this. Want to ask you if you’ve got a prediction for the upcoming season. Is it Warriors again? Where are you at? I guess where’s your heart at? And maybe what are the numbers telling you early?

Ben Alamar: Okay. So, we clearly have the Warriors. I think they are like 56-57% chance to win the championship, right now in our measures. So they’re far and away the best team. But if we look back on last season, we got to remember that, the Rockets were really just a Chris Paul hamstring injury away from winning the Western Conference and then likely the whole thing. So, there is some vulnerability there. So, I like Boston, I like Houston to make a run at Golden State this year a little bit, but still, if you had to put money down on, you got to go Golden State at this point.

Paul Michelman: Okay host Ben, what do you say? Did guest Ben make you a believer? Would NBA teams make fewer draft mistakes if they measured basketball IQ?

Ben Shields: You know, Paul, I think we’re getting there. I think Ben made a convincing case for a methodology to measure basketball IQ. I think to his point, we still need more and better data to effectively answer that question, but I think we’re getting there. So, I would say, yes, he did convince me that this is something to look at and to consider heavily going forward.

Paul Michelman: I think I’m largely a believer as well. I’m left with this one question about whether basketball IQ or basketball intelligence is just simply intelligence. I think Ben had an interesting take on that, but I’m still left not entirely sure that there’s a distinction. I really look forward to seeing more research and more data on that. Okay. So let’s bring in our producer, Mary Dooe.

Mary Dooe: Hey Paul. Hey Ben.

Paul Michelman: All right. Mary, what do you say? Thesis, yay or nay?

Mary Dooe: I’m going to go with yay. Now I’m not any kind of expert on basketball or the draft. However, I would say that I buy this argument. Obviously there are things that need to be done in terms of the actual measuring and the techniques to be doing that. But for me, I’m just thinking about sports in general. I was a competitive swimmer, and I think a lot of athletes always have this kind of X factor that you can’t quite explain. And so when I heard the guys talking about basketball IQ, I see it as sort of a way or a path to start defining and measuring what I see as an X factor. So, I don’t necessarily think it’s intelligence, but it’s a way to think and almost feel about a sport. And so in my opinion, you know, I buy it. That said, I’m not the person doing the NBA draft and losing the money to spend the extra time to do it. But I would say: Yeah, it would probably improve draft picks.

Paul Michelman: Thanks Mary. Thanks Ben. So we’re in agreement, three yay votes. And I think we’re also in agreement that we want more data and more information. Maybe we’ll bring back Ben a year from now and see how far we’ve come.

Ben Shields: All right. That wraps up this week’s episode of Counterpoints, the sports analytics podcast from MIT Sloan Management Review.

Paul Michelman: You can find us on iTunes, Google Play, and wherever fine podcasts are streamed. And please take a moment to post a review. We really want to hear your feedback.

Ben Shields: We’ll be posting new episodes every two weeks.

Paul Michelman: Counterpoints is produced by Mary Dooe. Our theme music was composed by Matt Reed. Our coordinating producer is Mackenzie Wise. Our crack researcher is Jake Menashi.