When NHL Best Practices Go Overboard

In pairing right-handed defensemen with left-handed defensemen, many NHL teams believe they are following a proven and, seemingly obvious, formula. If defensemen play on their strong side, they will be more effective at their jobs. Conventional wisdom on what works can codify with surprising speed in sports.

But when something becomes an accepted best practice, we tend to stop asking how it became so and, more dangerously, we often become blind to the evidence that the practice may not be best at all. Tyler Dellow, senior analyst at Counterpoints, believes this is precisely the case with defensemen pairings in the NHL.

Transcript

Paul Michelman: Join me, fellow travelers, on a journey to the spring of 1997. Deep Blue has done honor to computers across the globe by defeating Garry Kasparov in a rematch of their epic chess battle. The English Patient has been named Best Picture by the Academy of Motion Picture Arts and Sciences, undoubtedly the worst film ever to be so honored. J.K. Rowling is preparing to release Harry Potter on an unsuspecting world.

Meanwhile in Detroit, the Stanley Cup drought is going strong. Championship free for going on four decades, Hockeytown had come agonizingly close the previous two seasons, losing the Cup final in ’95, then setting the single-season wins record in 1996 but falling in the conference finals.

Detroit boasted two future Hall of Famers on defense in Paul Coffey and Nicklas Lidström, yet the pair of left-handers had floundered in the game’s biggest moments, Lidström especially. The not-yet Mr. Perfect was a minus-six in both the ’95 and ’96 series losses, meaning the Red Wings allowed six more goals than they scored while Lidström was on the ice.

The two lefties just couldn’t bring home the hardware, and by early in the 1996-97 campaign, Coffey was a Red Wing no more.

Ben Shields: Enter Larry Murphy. Murphy was a 35-year-old right-hander nearing the twilight of his career, but after moving to Detroit at the trade deadline in ’97, he would become an indispensable member of the Red Wings’ championship run. Paired with the lefty Lidström almost immediately, Murphy complemented the young Swede to a tee. Murphy was an NHL-best plus-16 in the ’97 playoffs, with Lidström close behind at plus-12.

The result? A 42-year Cup drought ended, followed by another title the following season. The two men continued to play almost exclusively together, with Lidström blossoming into a superstar.

As for Murphy, the back-to-back Cups must have felt something like déjà vu. As a member of the Pittsburg Penguins earlier that decade, he was the right-handed man to a different Hall of Fame lefty by the name of Paul Coffey. I’m Ben Shields.

Paul Michelman: I’m Paul Michelman, and this is Counterpoints, the sports analytics podcast from MIT Sloan Management Review. In this episode: Is the secret to a successful defense matching righties with lefties, or were the Red Wings simply the beneficiaries of Larry Murphy’s Law?

Paul Michelman: In Counterpoints, we look beyond the data in search of what the data reveals, or supposedly reveals, about what’s actually happening, both on the field and off. In each episode, we put one analytics-based hypothesis to the test and see how well it stands up.

Ben Shields: Today’s hypothesis: NHL teams have gone overboard matching right- and left-handed defensemen.

Paul Michelman: In pairing right-handers and left-handers, many NHL teams believe they are following a proven and seemingly obvious formula: If defensemen play on their strong side, they will be more effective at their jobs.

Conventional wisdom on what works can codify with surprising speed in sports, and in most other domains, for that matter. But when something becomes an accepted best practice, we tend to stop asking how it became so, and more dangerously, we often become blind to the evidence that the practice may not be best at all. And at least one expert observer believed this is precisely the case with defensemen pairings in the NHL.

Ben Shields: Which brings us to Tyler Dellow. An NHL columnist at The Athletic, he has been participating in the development of hockey analytics and writing about these ideas at various sites, including his own, mc79hockey.com, for more than a decade. He also worked in an analytics role with the Edmonton Oilers. You can put Dellow squarely in the camp of those that believe the NHL has gone overboard in their devotion to right- and left-handed pairings.

I spoke with Tyler about how left-right pairings became conventional wisdom, and what those who follow the strategy are missing.

Well, it’s not every day that we get to nerd out on hockey analytics, and Paul and I were just talking about [how] this thesis of handedness in defensemen pairings is overvalued. I wanted to start with a little bit of context here. I know that we have some hockey analytics fans as our audience members, but some may not be as familiar with this topic, so could you kind of give us a sense of the difference between right- and left-hand shots in hockey?

Tyler Dellow: Sure. It’s very simple. Basically, players are said to shoot either left or right. Basically, if your right hand is lower on your stick, you’re a right shot, and if your left hand is lower on your stick, you’re a left shot. It seems like it wouldn’t make a big difference, but it actually does impact how you handle the puck and how you see the ice. If you’re a right shot on the right side of the ice — there’s a lot of things that are easier than if you’re a left shot on the right side of the ice.

So, that’s kind of the first issue. Now, the second issue is there are more left shots than right shots, and so one of the challenges teams run into is finding right shots, both at forward and at defense. Over the past few years, there’s been a trend towards using more right shots on the defense courts.

Ben Shields: Why does this matter from a hockey strategy standpoint?

Tyler Dellow: Well, it’s easier to make plays on what they call your strong side. So for me, I’m a right shot; my strong side is the right side of the ice. If you imagine if the boards are to your right and your stick is to your right, it’s easier to reach out with your stick and, say, get a puck off the boards than if you’re on the other side of the ice, it’s a much more awkward maneuver.

So, one of the issues is that it’s easier to win pucks in a puck battle on your strong side. Another issue is it’s easier to make passes on your forehand if you’re on your strong side. So, if you’re a left shot and I’m a right shot and we’re moving up the ice together, if I’m on the right side, the way that my body kind of naturally positions itself, I’m looking out and I see the whole ice. If I’m on the left side, I’m kind of turned a little awkwardly in order to see the whole ice.

And this is an area that’s not entirely nailed down yet — precisely where all the differences come from — but it’s been pretty clearly established that players who shoot right, or if you have a defense pair and you have a right-shot player on it, it will do better, all other things being equal, than a defense pairing with two left shots on it.

Ben Shields: It seems like you believe that teams have gone a little bit too far in pairing right- and left-handed defensemen. Why do you believe that? What’s your core evidence as to why teams have gone too far in these areas?

Tyler Dellow: Well, so just to step back for a second, the research that’s been done — the first public article on this of which I’m aware is an article by a guy named Dom Galamini. He is with (I don’t know if I’ve said his name correctly) a site called Hockey Graphs, and he wrote an article setting out proof that when you’ve got a right- and a left-shot defenseman on the ice, you’ll do better both in terms of Corsi, which is shot attempts for versus shot attempts against, and you’ll do better in terms of expected goals. Expected goals is a relatively new metric in hockey, but the idea is that it sort of looks at the probability that a shot on net will produce a goal.

And basically he looked at what happens when there’s a right-left pair on the ice versus a right-right pair or a left-left pair, and he found that the results were better when you had a right-left pair than when you had a right-right pair or a left-left pair. So, what he inferred, and I think he’s right, is that the differences that I talked about in terms of it’s easier to make a pass or it’s easier to dig a puck off the boards, those differences actually show up in the results.

And so, that’s kind of the starting point, the evidence that you’re better with both a right and a left on the ice. Now, there’s a disproportionate supply of lefties because in Canada, people disproportionately shoot left. So, historically what you’ve seen in hockey is that left-shot defensemen make up a disproportionate share of the minutes for defense.

And what Dom showed in his article was that in 2007-08, about 50% of five-on-five ice time was played with a left-left pair. By 2015-16, that was down below 30%. I did a check this morning, and this year it’s down below 25%.

So, the question that raises is: Where are all of these right-shot defensemen coming from? My theory is that what’s happened is that coaches and general managers have started to weight a player’s handedness more heavily, and they’ve brought in players who otherwise might not be as strong as the left-shot players they’re displacing. And as a result, they’ve probably gone past the point of equilibrium, and the League should maybe pull back a bit on the obsessive need to have a left and a right on each pairing.

Ben Shields: So, basically what we’re hearing from you is that this notion of handedness may be a little bit more overvalued as a driver for performance of a defensemen pair.

Tyler Dellow: Right.

Ben Shields: What are some of the other performance factors that you think NHL teams should be prioritizing maybe over handedness?

Tyler Dellow: When you’re the coach or you’re the general manager and you’re making these decisions, you’re trying to balance how well does a guy make a play, how well does he defend, how well does he win puck battles, how well does he do all the skills of a defenseman? And then you’re trying to balance that against — I guess one of the things you have to consider is what hand is he — and I think there’s an argument that they’re putting too much weight at this point on the player’s handedness and not enough on the other skills that make up a defenseman’s value.

It’s really about striking a balance. It’s about finding that balance between what the defenseman does and his hand, which will impact on the results.

One of the things that’s really interesting about this topic is that it’s a topic that flows from coaches. Mike Babcock, who’s the head coach of the Canadian men’s national team and the Toronto Maple Leafs — he’s talked a lot about this publicly, how he likes a left defenseman and a right defenseman. And that’s kind of — his discussion of it has kind of gone hand in hand with how the League has changed towards wanting these right-shot defensemen.

I think it’s really a challenge for managers and coaches to strike a balance with limited information.

Ben Shields: Let’s pick up on that point about limited information because, look, you’ve been in hockey analytics now for more than a decade. You are a pioneer here. Do we have the right data in hockey right now to get at the most important skills for defensemen? And if not, what needs to change in order to equip coaches and general managers with better information in order to make better decisions?

Tyler Dellow: The first thing that you would want to do to really accurately answer this question is to understand just how big the difference the handedness impact is when it comes to picking a puck off the board or making a play or making a pass on your strong side, and if the handedness impact shows up in other parts of the game. And it may well. One example is, taking a one-timer is much easier if you’re on your, what they call, weak side. So, there actually might be some areas where a left-shot defenseman on his right side has an advantage.

And so, if you had, for example, some tracking data and you could start to get into more detailed analyses of puck battles won, and is the player taking more high-value shots, and is he able to make better passes on his strong side, you could start to kind of quantify the size of the effect of handedness on the performance of left- and right-shot defensemen. Once you were doing that, then you could start to push into, well, are we better off with a right-shot defenseman who’s objectively not as good, or are we better off with a left-shot defenseman who’s better?

Different sports have had to deal with this challenge. This is no different than what baseball was dealing with 30 years ago when Bill James was writing about platoon effects and whether or not you should prefer to have a stronger right-handed pitcher on the mound at a given time or a weaker left-handed pitcher. And just the challenge that hockey presents as compared to baseball is that baseball has a much cleaner record of what’s happened than what hockey’s historically had, where we’re having to kind of infer the impact of handedness from shot data or from expected-goals models, as opposed to being able to kind of directly measure the skills that are impacted by it.

Ben Shields: Is player tracking data the missing link here?

Tyler Dellow: I think it’s going to be a huge step forward. There’s people who think that it will be a magic bullet, and it won’t be that, and you’re going to have to do a lot of work to, I think, really suck good insights out of it. But I think for those teams or those people who ask good questions and are able to structure their studies correctly, I think they’ll be able to pull an immense amount out of it.

And I think if you look at other leagues — basketball’s one that really sticks out to me — they’ve done a lot with the player tracking data, and they’re able to measure individual confrontations between players in a way that hockey hasn’t been able to do yet.

And so, I think it will be tremendously useful for assessing some of these questions. And again, what’s interesting about this is that this was a point that coaches were making. One of the best coaches was saying this, and then Dom was able to show that the data backed it up. And once we get this next-generation data, you’ll be able to dig deeper into it and start to put a price on it, and once you can do that, then you can really accurately answer whether or not coaches have gone too far in preferring to have right-handed defensemen.

Ben Shields: All right, so definitely the jury sounds like it’s still out on this. Player tracking data is going to help when it does come to the NHL. And by the way, do you know when that’s happening?

Tyler Dellow: It increasingly sounds like 2019-20 will be the season. We’ve heard that a lot, but it seems to me like there’s some real momentum this time, and hopefully they get that there and they make a lot of it available. Because I think what you’ve seen in other sports — I guess particularly baseball here — is that when you make the data available, you get the benefit of the amateur analyst community, and a lot of good stuff came out of that in baseball, and I have no doubt a lot of great stuff would come out in hockey, if they did it.

Ben Shields: So, when we look at the season this year, can you give us and our listeners a defensemen pairing or two that we should pay attention to?

Tyler Dellow: That’s a great question. I think San Jose is probably the best team because it’s got a bunch of really interesting pairings. They’ve got Brent Burns and Marc-Édouard Vlasic and Erik Karlsson. Those three players, you’ve got two guys who’ve won Norris Trophies, and Vlasic might be the best defenseman of the three and he’s never won it. So, it’s a very interesting mix of defensemen, and they’re trying to find a way right now to maximize the value they get from all three of them.

And so for me, if you’re interested in a group of defensemen to study, those are the ones you want to pay attention to, to see how the coach tries to maximize value from them. It’s interesting, Pete DeBoer is a fascinating guy. He’s the head coach there, and I actually wrote about this last year, but he is so aggressive in trying to get his matchups with Vlasic against the other teams’ best forwards that he’ll just have Vlasic — he’ll put him up for a face-off, and if the best players don’t come on, Vlasic just leaves the ice as soon as it’s over.

I think if you’re looking for what might be the next step in defensemen usage, San Jose is a good team to keep an eye on.

Ben Shields: Anything else beyond defensemen pairings that teams should be paying attention to with regard to adopting innovative strategies based on analytics? Especially with the rollout of player tracking data, hopefully, in the future. Anything else regarding hockey analytics that teams should really be paying attention to, from your standpoint?

Tyler Dellow: Yeah, I think what we’re going to see is teams are going to be able to much more effectively evaluate what works and what doesn’t. What I think is hard to kind of wrap your head around is it’s hard to really know what’s going on around the league. I’ve talked to friends in TV who will have, say, a coach in for the playoffs or something whose team didn’t make the playoffs, and they say the coaches are pretty open that they have no idea what’s going on in the other conference. And it’s because you’re really focused on your team, and you only see the other team once or twice a year, so you have no idea what’s going on there.

So what I think that the data will enable the teams that are aggressive and smart about it to do is get out in front of, yes, we can use this data to sort of create descriptions of all 31 teams in the league, and we can understand what everybody’s doing, and we can start to look for best practices that we can steal. And [what] to me is going to be a really fascinating thing is which teams are able to quickly identify what they should take from other teams and what works and what doesn’t, because I think that’s hard to do right now just working with video and the limited data the NHL provides. And with the tracking data, I think you’ll be able to take an enormous step ahead in terms of doing that.

Ben Shields: I agree, and that tracking data can’t come here soon enough.

Tyler Dellow: Yes.

Ben Shields: Do you think that the advent of more advanced hockey analytics will make the game more fun for fans to watch?

Tyler Dellow: You know what? This is a great question. I have a friend who’s in media, and he always moans about how baseball has been adversely impacted by data because teams are shifting and batting averages are down, and it’s just become strikeouts, home runs, and walks. The counterargument I give is that basketball’s wild now. It’s all just three-point shots and it’s high scoring and it’s super fun.

It’s interesting, one of the things you’ll see in hockey — and actually, I wrote about this at theathletic.com — is talking about how power plays are better than ever. What you’ve seen is that teams have shifted from using three forwards and a defenseman to using four forwards and a defenseman, and I really think that that’s been, to some degree, driven by data. Analytics people have been pushing for this for 15 years. That to me is an example.

And, goals are fun. I’m a huge believer in that. There’s a lot of old-timey purists who like to say, “Oh, a 2-1 game, that’s what you want.” I don’t buy that. I think hockey’s best when it’s a 6-5 game.

So, that to me is an example of the data made the case, and between the data and coaches seeing other teams doing it, we suddenly got more goals and more excitement. So, I absolutely think that data can make the game better, and I hope that when it comes on board, I hope that coaches and analytics departments try to use it to create just as much as they try — do use it to destroy what the other team’s doing.

Ben Shields: All right, coming back from that discussion with Tyler. Paul, Mary, what did you think?

Paul Michelman: I think Tyler mounts a really interesting argument that ties into, I think, a really interesting larger issue. What Tyler is demonstrating is how important it is for organizations in sports and otherwise to pause every once in a while and question why they do things the way they do. Organizations only tend to do this when things are going wrong; we’ve got a problem, what led to this problem? How can we do things better? But when they wait until things are going poorly, teams miss the opportunity to improve and to get ahead of the competition.

And I think righty-lefty pairings is one of these cases, at least based on what Tyler has said. NHL teams have this slavish devotion to a practice. This practice has gone on for a long time, so they believe it works.

So I’m not in a position to say it doesn’t work. I think teams have evidence that it is a pretty good practice. But is it always the best practice? We don’t know until we really look at a more detailed level as to what is driving successful defensemen pairings’ performance. So, your pairing is plus-20 so far this season. That’s great. Maybe it could be plus-30.

It’s incredibly valuable for a team to step back — any organization to step back — every once in a while and question their own success. If they do the investigation, and they affirm that there’s very good reason to do a practice a certain way, keep doing it. Go all in. But you’d be surprised what you discover.

And I think Tyler’s argument is a great call to organizations to take a breath and question their own success.

Ben Shields: Paul, the problem is you have to have data to do a rigorous investigation on why you are having the success that you’re having. I know a lot of NHL teams would like to take that step back and do a sophisticated analysis, but as we discovered in the interview with Tyler, we are still a ways away until player tracking data makes it to the NHL. Now, hopefully that will be coming soon, and when it does, mark my words, we will be seeing an analytics revolution in hockey where the teams that invest in analytics will be able to find creative, competitive edges like they’ve never found before.

So, for all the hockey fans out there, analytics experts out there, we are on the cusp of creative and innovative analytics work in hockey just as soon as that player tracking data arrives.

Mary, your thoughts?

Mary Dooe: Well, I’m not so sure about the thesis or the argument, but I would say that at the very least, the three of you have convinced me that hockey is about a lot more than guys fighting on ice, and I might just catch a couple games this year. And, I like The English Patient.

Paul Michelman: This has been Counterpoints, the sports analytics podcast from MIT Sloan Management Review.

Ben Shields: You can find us on iTunes, Google Play, and wherever fine podcasts are streamed. And please take a moment to post a review. We really want to hear your feedback.

Paul Michelman: Counterpoints is produced by Mary Dooe. Our theme music was composed by Matt Reed. Our coordinating producer is Mackenzie Wise. Our crack researcher is Jake Menashi.