What the NBA Gets Wrong About Lottery Pick Protections

In this episode, we take a closer look at the value of pick protections in the NBA draft — and how your favorite NBA just might be doing it all wrong. The NBA draft is all about value: Just a couple of selections higher or lower could be the difference between a franchise-altering superstar or another half-dozen seasons selecting in the lottery. But when it comes time to move these assets around, value sometimes gets thrown out of the window, and teams make deals involving pick protections they later regret. To help us understand why — and to chart a better strategy for pick protections — we speak with Ben Foster who presented his and Michael Binns’s research on valuing protections of NBA draft picks at the 2019 MIT Sloan Sports Analytics Conference.

Transcript

Paul Michelman: The first era of the Brooklyn Nets will always be defined by maybe the most lopsided trade in NBA history. Looking to hit the instant contender button for the 2012-2013 season, the Nets sent a king’s ransom to the Boston Celtics for aging superstars Kevin Garnett and Paul Pierce. It, of course, backfired spectacularly. Brooklyn made a solitary second-round playoff appearance, while the Celtics enjoyed a bevy of lottery picks. It set the Brooklyn franchise back for years, but it might have been completely avoidable if not for another Nets’ trade the season prior. If only the team had been a little less reckless when it came to lottery protections.

Ben Shields: Then still in New Jersey, the Nets knew that their upcoming move to the big city needed some star power. And to attract star power, you need cap space. A deal was struck with Portland at the 2012 trade deadline that would send a few salary-eating players out West, along with the Nets’ upcoming first-round pick. The Nets did what they thought was their due diligence and put top-3 protection on the selection — meaning that if the pick were first, second, or third, the Nets would hold onto it for another season. Confident that the draft was so top-heavy that such limited protection was all that was necessary, the Nets completed the deal, and while the pick didn’t land in the top three, it still wound up sixth. Turns out that the sixth pick still carries a lot of value. And in this case, that value turned out to be four-time All-Star and Portland icon, Damian Lillard. In another universe, maybe one of the magical moments Dame has produced in his career would have belonged to Brooklyn, as well. Instead, the Nets’ decision had led to one of the game’s great what-might-have-beens. I’m Ben Shields.

Paul Michelman: I’m Paul Michelman, and this is Counterpoints, the sports analytics podcast from MIT Sloan Management Review. In this episode, we’re taking a closer look at the value of pick protections in the NBA draft and how your favorite team just might be doing it all wrong.

Ben Shields: The NBA draft is all about value. Just a couple of selections higher or lower could be the difference between a franchise-altering superstar or another half-dozen seasons selecting in the lottery. The importance of having dynamic players on entry-level contracts means a high draft pick might be the most valuable asset a team could own.

Paul Michelman: But when it comes time to move these assets around, value sometimes gets thrown out the window. It’s easy enough to place a top-10, top-5, or even top-1 protection on the draft pick and toss it into a trade — out of sight, out of mind — so to say. But when that pick eventually conveys, just how often is the team that gave the pick up or agreed to the specific protections regretting their decision? To help address that question, I spoke with Ben Foster who presented his research on valuing protections of NBA draft picks at the 2019 Sloan Sports Analytics Conference. Ben, thanks for joining the program.

Ben Foster: Hey Paul. Thanks for having me, and I’m glad to continue the long line of Bens that you’ve interviewed for the podcast.

Paul Michelman: This actually will be the last Ben. I’m putting a ban on that after today. So let’s talk about something more interesting than Bens. Let’s talk about your research. What prompted you to look at NBA draft protections in the first place? What was the moment of inspiration?

Ben Foster: So I was probably sitting on a couch. I’m a graduate student, and I was a graduate student back in 2015 following the NBA and following the trade deadline. And I don’t know if you remember the 2015 trade deadline, but there were a lot of trades and a lot of those involved draft picks. And a lot of those draft picks had protections, and there were things like lottery protected and top-10 protected. And I think I remember saying to my roommate at the time, who was a coauthor on the paper, “That seems really odd that all of these pick protections keep coming out at round numbers.” So we combed together the last five or 10 years of traded NBA draft picks and plotted out the number of picks that had each level of protection. And what we noticed is that lots and lots of NBA draft picks are protected at top 5, top 10, top 14. There’s a few top 3, and there’s not really many in between — maybe one or two, mostly zero at all those other levels. And this struck me as likely inefficient, and so we decided to kind of jump in and see if we could kind of think about why that might be the case. And then we designed a tool to try to value more precisely how much draft pick protections were worth.

Paul Michelman: Well, great. So let’s go at this kind of one piece at a time. When you reveal what the data found, it’s one of those kind of surprising/not surprising things, right? Surprising that there is such a concentration. But when you look at where the concentrations are – top 3, 5, 10, these kind of natural sets that are familiar to us; 14, because that’s the number of players in the lottery — [it] kind of makes sense. But I’m curious how this connects to other research. Did you draw on other research? Did you kind of vet this against other studies outside of sports analytics?

Ben Foster: Yeah, so we ended up employing a lot of tools from systems modeling and from finance to come up with our model. We operated under the hypothesis that at least part of the problem might be imperfect information or imprecise information. GMs don’t have a way of precisely valuing how pick protections change the value of an NBA draft pick. And as such, we built a model to try to contribute some information there. Whether that information becomes private information and is used to a single team’s advantage or is public information is another interesting discussion. But there are other reasons you could think of why teams end up on these round numbers. I’ve thought of this kind of outside of the scope of our actual research, but I think there’s quite a bit of evidence in managerial economics and finance that managers sometimes will operate in ways that aren’t optimal for the firm or organization, because their incentive structures lead them to make much more risk-averse decisions than perhaps the firm or organization desires. And so I’ve thought about that as we consider further steps, future steps with our work. One reason teams might be ending up on these round numbers is just because the GMs are somewhat risk averse and don’t want to do something kind of outside of the norm.

Paul Michelman: Intuitively, it’s easy to agree with that point. And, of course, we’ve seen lots of other evidence of that in the management of sports. Right? Absent actual data to drive a decision, GMs are risk averse and then also tend to follow kind of a herd mentality — which doesn’t make them a special class of managers. We see that across management, as you’ve said.

Ben Foster: Right.

Paul Michelman: So let’s turn to the tool. Talk to me about how you built the tool, what it does, and what you found.

Ben Foster: Yeah, so the idea here was to see if we could price or find the value of draft pick protections. There’s lots of ways you could conceive of doing that. One way is to value NBA draft picks and then value NBA draft picks with protections on them, and then take the difference of those two things. And that helps you identify the value add or subtract of the protection itself. So quickly, we went into kind of building out a tool that can value or put some value on an NBA draft pick, a future NBA draft pick for a specific team and a specific round of the draft. And then we built out mechanisms for adding in protections. And so to do this, you have to account for lots of future uncertainties about quite a few different things.

So there’s three, I think three, main sources of uncertainty that we had to capture. So the first is near-future team performance. So if you imagine a trade at the trade deadline — for example, there are 20-some-odd games left for every team in the NBA, and we wanted to capture the likelihood that the team that traded away a pick would end up in a particular position at the end of the year in the standings, because that’s going determine either what pick you get or where you end up in the lottery order. So that was one piece of team performance that we needed to capture. The other piece of team performance is future year performance. So pick protections, obviously, can roll out way into the future. And so a pick that’s protected, say top 5 – if the pick ends up in the top 5, it often rolls over to the next year. And we needed to figure out how likely a team was to finish in a particular spot in subsequent years. And so there’s a different method for doing that.

And then the third source of uncertainty is how good is a player going to be that ends up being picked with that particular pick asset. So we looked at historic data of players and where they were picked in the draft to try to pull out some information about… You get the first pick, it may be Anthony Bennett, it may be LeBron James, so you’re not guaranteed a certain value out of any particular pick position. So we pulled all those things together into a Monte Carlo simulation format that allowed us to combine all that uncertainty into a single distribution of possible outcomes. We measured them using win shares. You could use kind of whatever player value metric you want. Win shares is fairly easy to interpret it, and it was easy for us to get data on, which is why we used it. So we end up with this distribution, and then we put that into a financial asset pricing model that allowed for us to account for some things like risk preferences and different types of asset classes, and allowed us to boil it down into a single kind of win shares number that we assign as the value of the pick. And along the way, we incorporate whether it’s protected or not, and what levels it’s protected.

Paul Michelman: So let’s talk about the win shares for a minute. Where are (or are there) kind of natural consistent points of differentiation? Like win shares among the top 3, there’s a break; or the top 6, there’s a break. So where did you find kind of the key differentiators?

Ben Foster: Yes. So there’s quite a bit of noise in that data we were using in the past, I think 30-some-odd years of information. So we’ve only got 30 picks at each level. So we’ve got a lot of things going on. So what you find when we plot out the probability distributions of win shares generated by a given pick position, is that a lot of picks will cross each other. So suddenly the third pick often looks a lot better than the second pick. And so we had to do some things to kind of smooth out those attributes. Because we didn’t want our model… to magically reward getting the ninth pick, for example. The ninth pick looks a lot better a lot of the time than the sixth, seventh, or eighth pick, at least in terms of player performance. So we did some things, used some statistical methods to smooth out those distributions, and then we grouped them together into a few categories of what we call asset classes in the paper.

So we identified seven different asset classes. So those are groups of picks that seem to perform similarly, and those ended up being: Pick one kind of stands by itself. Pick ones tend to perform a lot better than all the other picks. And then two and three look pretty similar as do four and five, and then the groups tend to get a little bigger. So six through 10 is pretty similar. And then it’s 11 through 18, 19 through 24 and 25 through 30. So when we put that into our modeling framework, we end up assigning different risk preferences to each of those asset classes that we sort of calibrated or tried to calibrate using some historic data, although we had some plans to kind of beef up those methodologies in calibrations.

Paul Michelman: So when you look at those asset classes, how does the value of protecting any one of those classes get affected by the number of years out where that protection is going to be applied?

Ben Foster: Yeah, so that depends a lot on where the team — how good the team is now. So as we look forward into the future, and the further we look forward in the future, we kind of see the uncertainty or the variability in outcomes spread out quite a bit. It’s hard to say what, say, the Milwaukee Bucks are going to be like in five years. If you had to bet where they would finish in the NBA, that’d be a tough assessment to make. And so this uncertainty kind of tends towards teams kind of migrating towards the middle and reduces the likelihood of the extreme outcomes. So really good teams this year, we don’t necessarily expect to keep being good five years out into the future. And so the protections tend to kind of converge on something that’s universally good out into the future, or it’s good for kind of all teams. And what it turns out to be, is there’s a really high value in protecting, say top-1 protected, because that first pick is so valuable to teams. It’s consistently so much better than the players picked in the later picks. And so those are players that kind of change your franchise. And so by protecting top 1, you kind of recover a lot of value in the traded-away pick, especially as years extend out.

Paul Michelman: What about the projected quality and depth of the draft itself? Is that a factor?

Ben Foster: Yes, it would be a big one if you’re using this as an actual decision-making tool. And we’re working on ways to try to change the structure of the tool such that folks can use it more readily in the decision-making process. We do not account for projected draft class quality — because I’m not really a basketball…. There’s lots of people that are much smarter about basketball than I am and probably have much better assessments, and their teams have much better assessments of these things. So we didn’t attempt to do that — though it wouldn’t be hard to add to adjust kind of the performance you might expect out of the different pick positions.

Paul Michelman: So Ben, should teams factor in salary ramifications when valuing protections? The No. 1 pick is going to cost something like two-and-a-half times what the No. 10 pick will cost in that initial three-year rookie deal.

Ben Foster: Yeah, that’s definitely a factor. And I think we’ve captured that in some of our risk preference calculations for the different asset classes. But you would maybe consider those things as kind of an additional factor as you’re negotiating a trade. We envision this tool as useful for — perhaps it’s a GM who’s negotiating a trade and wants to determine an optimal place to set protections based on the player they’re trying to trade for or whatever the trade assets that are involved. One consideration from the team’s perspective is going to be salary. We built this tool kind of generically for any team. Obviously, if you’re using this in a real decision-making capacity, there’d be lots of other factors related to your team that you’d want to capture. But we do capture, I think, some of those salary implications in our risk preference part of the model.

Paul Michelman: So is it possible that the whole practice of protecting draft picks is just fraught? That there are so many variables and so much unpredictability that it might be better not to do it at all?

Ben Foster: There is a lot of unpredictability, I would say, and I think that’s increased with the new lottery probabilities. The lottery just happened not too many weeks ago, and there’s a lot of discussion about how interesting it was from a kind of viewer perspective because lots of teams were moved around. It was speculated afterwards by a variety of folks that this increase in uncertainty and variability in outcome would lead to kind of a chilled market for protecting picks or something like every pick would be traded with de facto top-5 protection.

I actually think it should act in the opposite way. And I’ll try to make this make sense. But if you imagine the role that pick protections are playing in a trade — they are this asset, this thing that used to be one asset (which is the draft pick, say, the 2020 first-round draft pick) that now can take on all sorts of values based on the different protections that you put on it. So if you’re negotiating for a particular player, and you’re trying to come up with some package of assets that’s of equal value to that player, this draft pick — a draft pick with your ability to protect it — becomes really, really valuable. Because you can perfectly tailor — theoretically you could perfectly tailor your asset package to the value of that player. As variability in outcomes goes up, the values you could change that pick to also becomes higher. You end up with a bigger range of possible values your pick could take on, and that, theoretically, should facilitate more trading or more likelihood that you could end up with an asset package of equal value.

Now, with that said, I don’t know that I’d trust that to actually happen. I would expect, especially given our previous discussion about other reasons why teams might be protecting picks at round numbers; namely, risk aversion of GMs or other factors, that this might actually chill the market, at least initially. It’ll be interesting to see how it plays out. I have another kind of speculative thought that pick protections might be regulated away, or at least in large part regulated away, by the NBA at some point. There’s lots of NBA regulations that… trade. And one of the things that the teams still have are these pick protections that ought to enable more trading. But as it gets kind of crazier and crazier, I could imagine the NBA not wanting fans of teams to be going to the lottery and having, say, six different slices of a bunch of other teams’ picks to not really knowing what might happen. It might get quite complicated. Although, maybe that would make the lottery show more exciting and have better viewership. But anyways, there’s a lot kind of at play. And yes, I do think that the change in variability or the increase in variability of outcomes might chill the market in practice, but theoretically, I would actually have expected it to make pick protections a more valuable practice.

Paul Michelman: So Ben, I want to come back to those kind of key breakpoints in win shares. Please correct me if I [don’t] have this correct. There’s a break after 1, a break after 3, a break after 5, and a break after 10. Is that correct?

Ben Foster: Yes, that’s correct.

Paul Michelman: So what’s really interesting, those are all “round numbers” when it comes to our thinking, right? Those actually don’t vary wildly from the current practice, which was, on the one hand, interesting. But it also suggests a bit that for the team that is accepting the protection, it feels like this model has some near-term things where a team could exploit this.

Ben Foster: Yeah. So when we discovered this, I thought: Well, there’s part of our answer, and that might explain why these picks are kind of clustering, protections are clustering at these numbers. I still think that just assigns what risk preference value we put in the model. There’s still a bit of variation between the different pick numbers. Six is still better than pick 10, for example. One thing I think we have figured out in pursuing this is that there are probably a lot of assets out there that are under- or overvalued or misvalued in some way, and smart teams can pick up on that. We kind of mention in the paper there might be “arbitrage opportunities” — that might be stating it a little bit too positively. It’s difficult to arbitrage a market that’s relatively illiquid. But there would be, perhaps, some low-hanging fruit. And you’ve seen some teams in recent years who’ve started to assign more and more complicated pick protections on the picks they trade away. And I think that’s evidence that those teams are probably thinking a little bit more seriously about what pick protections do and how valuable they are.

Paul Michelman: So if you were to continue this line of research, what would be the next kind of set of data or element of the tool that you’d be pursuing?

Ben Foster: Yeah, so we’ve conceived of this not quite as a proof of concept. It’s a couple of layers beyond that. But we had relatively limited resources doing this on a laptop on the side. It’s not my or my research collaborators’ full-time jobs. So we chose methods that were relatively easy to implement and perhaps aren’t the most precise or the most accurate. So there’s a couple ways we would improve those, including ways we characterize various bits of uncertainty. I think I also mentioned that we’d like to build the tool out so that it’s a little bit more, kind of ready to be used in a decision-making capacity, so that it would include things like toggles for team-specific risk preferences or feelings. So things about draft class quality. Perhaps it wouldn’t be too hard, I don’t think, to add some toggles that allow you to adjust some of these distributions so that they align with expectations. For example, this year, you might be adjusting that first pick out a little bit and making it a little bit more valuable. And the other (the second, third, fourth, and fifth) picks back a little bit. I think a lot of folks think it’s a relatively top-heavy draft. And then kind of beyond this specific tool, I think we’ve thought of a lot of opportunities to think about nonplayer assets in the NBA and try to come up with valuation models for those things. There’s not a lot of attention on nonplayer assets. There’s relative dearth of data there, which might be one reason, and it’s also not kind of the main player. It’s kind of fun to study players and coaches ’cause that’s leading to winning. But studying how GMs make trades and operate and sign contracts is a little bit less fun. But I think there’d be lots of opportunity to think about these nonplayer assets, things like Bird rights and contract structures, and try to find systematic ways to quantify their value.

Paul Michelman: There’s been a long discussion about moral hazard in the NBA with respect to draft position and teams tanking to get the lowest draft pick possible, or in the age of the lottery, to get the highest number of Ping-Pong balls in the lottery. How does moral hazard play out with protecting picks?

Ben Foster: Yeah, so I imagine it’s kind of the same phenomenon we see with teams trying to get better lottery odds. That once a pick is protected — for example, the Lakers in 2015 traded away, or in 2013 they traded away a 2015 pick that was top-5 protected. They would’ve had a lot of incentive in 2015 to stay within that top-5 protection so that they could keep their pick. And so we’ve discussed how we might try to capture this additional incentive to keep losing. And we don’t actually capture this explicitly in our model, and we’re thinking about ways of addressing it. I mean, one thing that works in our favor is that every team around the worst five teams in the league has incentive to lose at the end of the year. And so perhaps it wouldn’t change the distribution of outcomes all that much that the Lakers have slightly more incentive to keep losing. But the other thing that I’ve long wondered is when some enterprising GM is going to trade away a pick — where he trades away all the even numbers to one team and all the odd numbers to another team. Theoretically, kind of eliminating his own moral hazard to behave in a certain way. And I think that would be a highly unlikely scenario but a very interesting and cool one if there are any GMs out there who want to do that. I think that would be quite fun.

Paul Michelman: Ben Foster, thank you very much.

Ben Foster: Thanks, Paul.

Paul Michelman: This has been Counterpoints, the sports analytics podcast from MIT Sloan Management Review.

Ben Shields: You can find us on Apple Podcasts, Google Play, Stitcher, Spotify, and wherever fine podcasts are streamed. If you have an idea for a topic we should cover or a guest we should invite, please drop us a line at counterpoints@mit.edu.

Ben Shields: Counterpoints is produced by Mary Dooe. Our theme music was composed by Matt Reed. Our coordinating producer is Mackenzie Wise. Our crack researcher is Jake Manashi, and our maven of marketing is Desiree Barry.