“Figures often beguile me…There are three kinds of lies: lies, damned lies, and statistics.” – Mark Twain
Hockey statistics lie.
Before my much anticipated lynching, allow me to clarify my position. While they may be fun to discuss, advanced statistics fail to tell a complete and accurate story about a game, as well as a team. Saturday’s Avalanche game against the Montreal Canadiens presents a prime case.
The Colorado Avalanche, seventh place in the Central Division, faced the league leading Montreal Canadiens at Bell Center. If one missed the first period and merely looked at the statistics, the information made no sense. The Avalanche led 3-1 on only eight shots against the Canadiens’ 18 shots. The shot count alone would indicate the Avalanche were dominated throughout the first period and that somehow the Avalanche managed to just get off a couple of lucky shots. Looking at the statistical models on War on Ice, the Avalanche appeared dominated.
However, watching the game provided a very different experience. The game showed lots of back and forth motion. Possession between the teams appeared relatively even, with Montreal getting more dangerous shots in bunches, while the Avalanche countered, only having to really fight for positioning. But the statistics told a different story.
I decided to re-watch the first period, keeping track of only two things: the amount of time the puck stayed in the neutral zone or Avalanche’s offensive zone and the number of shots from near the blue line for both teams. Despite the high number of shots logged against Avalanche goaltender Reto Berra (leading one to think the Canadiens had set up camp outside of the Avalanche net), the puck actually spent 10:39 in neutral territory and the Montreal end of the ice, despite the Avalanche having the only penalty kill of the period.
As for blue line area shots, the Avalanche threw only three shots from there on net, while the Canadiens managed more than ten. There are two ways to look at the blue line shots. One, the Avalanche defense kept the Canadiens from getting high percentage shots down low by the net. Two, the Avalanche need to put more shots on net. Yet, the Avalanche scored well in the first period (three goals), so it’s hard to fault their shot choices.
This exemplifies the fault within most modern day hockey statistics. Many of the calculations center on shots taken, blocked shots, shots that went wide, and scoring chances. The challenge? These are all subjective. There is an official scorer who determines what qualifies as a shot or a scoring chance. However, during the preseason when there were no official scorers, even between television stations showing the same game, there were marked differences in shot totals.
During the broadcast of the first period of Saturday’s game, at one point the official scoring showed the Avalanche had four shots compared to twelve for the Canadiens. Nearly five playing minutes later, the official scoring showed the Avalanche only had three shots. Marc Moser, play-by-play announcer for Altitude Radio, even wondered how the Avalanche could have lost a shot in the game. Obviously, there’s plenty of room for error with counting shots.
Shots on goal are technically supposed to be counted any time the puck approaches the net in such a way that it would score a goal if it wasn’t touched by the goaltender. Shots that are deflected or blocked, hit a goalpost, or sail wide are not counted as shots on goal, but fall into different subgroups such as Blocked Shots. However, what qualifies as a shot seems to vary. When I counted the Avalanche’s shots in the first period, I came up with ten shots on goal: three ended in goals, four sailed wide, and two were blocked. It became very clear shot count was subjective. Yet even that doesn’t tell the whole story.
One of the impressive aspects of Saturday’s game centered on the number of Avalanche players fighting for possession in the neutral zone. Avalanche players sprawled on the ice to prevent the Canadiens from interrupting an Avalanche drive into the offensive zone. Others battled to keep the puck and pushed opposing players off the play. I saw two Avalanche players dive to block offensive turnovers. Colorado showed they could be tough in fighting for possession. Montreal, one of the best teams in the NHL, contested Colorado’s pressure and tipped lots of passes.
Neither of these efforts counted in statistical analysis, especially because those actions happened in the neutral zone. Effective neutral zone positioning functions as a key to some teams’ strategy, yet most statistical measurements overlook neutral zone efforts and focus instead on shots.
Corsi, the much touted statistical measurement for teams and players which is based on shots (i.e., attempted, blocked), showed some very interesting data concerning the Avalanche/Canadiens game. According to War on Ice, Blake Comeau suffered one of the worst ratings among Avalanche players in terms of Corsi measurements, yet he was one of the more impressive players in the game(see Avs Game 17 Grades).
Also, according to the Corsi rankings, Colorado defensemen Erik Johnson and Francois Beauchemin ranked similarly as Comeau even though they both had a plus four rating (meaning the team netted four points over the time they were on the ice). Of the Avalanche defensemen, Brandon Gormley had the best Corsi number. While Gormley adds good value for the Avalanche, I don’t know of anyone who thinks he should have the top pairing minutes of Beauchemin or Johnson at this juncture.
Only three Avalanche players ended up in positive Corsi numbers for the game: forwards Matt Duchene, Nathan McKinnon, and Mikhail Grigorenko. Corsi analysis fails to allocate appropriate values for good defensive effort, and while shots are one measurement tool, they tell an incomplete story. A player’s numbers lack the ability to accurately reflect the level of competition they played against. For example, the first line defensive pairing will frequently encounter the best scoring line for the opposing team, skewing their ratings.
The biggest challenge with statistical analysis for hockey relates to the ability to manipulate the numbers to support any argument. Corsi analysis works best for evaluating performance over the long term. However, one could make the argument, after looking at the cumulative Corsi numbers for the Chicago Blackhawks and the Los Angeles Kings for the past five years, the teams’ play dropped off during the playoffs. One could also argue that their regular season statistics weren’t a reliable indicator of whether they would win the Stanley Cup. In fact, last year the Los Angeles Kings’ Corsi numbers charted excellently, yet the team failed to make the playoffs. There’s a large grey area around both the interpretation and the proper application of advanced statistics
The ultimate number to consider has been, and will remain, wins verses losses. That’s the only statistic that matters.
Hockey statistical analysis has a long way to go before it can really give an accurate assessment of a team, a game, or even a player. The analytics for the Montreal/Colorado game tell a very different story from what one saw watching the actual game. Maybe it’s because hockey is such a complex, fast-moving game, with lots of variables; math fails to accurately represent the events. While baseball has had over 150 years to develop it’s statistics—and still struggles to accurately reflect the course of a game and/or a season—hockey analytics is still in its infancy. Hockey needs more time to develop a better way of evaluating teams’ and players’ levels of play. Until then, people need to remember that the numbers may be fun, but they can’t compare to actually watching the game.
0 Comments (1 conversation)
Boy…I get why a lot of Avs fans are really anti-analytics, I really do. It doesn’t change their value or the fact that, no statistics don’t lie. They are literally incapable of doing so. The numbers are what the numbers are. People might try to spin them one way or another, but that’s simply not what’s happening with advanced stats in hockey – the stats are all there, they exist as the do, and they are what currently hold the bar-none strongest correlation with things like winning percentage and 5v5 scoring – and they have over a decade of NHL statistical history that backs them.
This article is just chock full of every disproved anti-analytics platitude out there, the most egregious of which being the idea of using a single game as an example of why analytics don’t work. That’s because they DON’T WORK THAT WAY. Analytics work over seasons and years and tell us what, over a decade+ worth of data, teams that win are doing more often than not statistically – and that’s taking more shot and giving up less shots (shocking!)
It doesn’t mean you never have games where a team that shot less and gave a ton of shots up wins, but on the whole winning and 5v5 scoring are much more strongly correlated strong possession metrics.
Regular Season Score adjusted CF% (all situations) of every Cup winner since 2005-2006
Canes – 50.1
Ducks – 55.4
Wings – 60.0
Pens – 48.3
Hawks – 57.6
Bruins – 51.5
Kings – 53.8
Hawks – 55.0
Kings – 56.2
Hawks – 54.5
So aside from a Pens team with Crosby/Malkin on it, every cup team in the last decade has had +50% score adjusted corsi which has the strongest correlation with winning% and 5v5 goal scoring.
It’s not just a coincidence, there are data to back this stuff up, and a single game where it doesn’t go according to the perceived script of those that don’t get how the data works doesn’t chance that.
Well, it seems you have an idea of what you are talking about, but fail to see the crucial element of numbers and statistics in general. Numbers (and statistics) might not lie, but what qualifies a tally mark does. Hockey, at its core, is a fluid, dynamic game with quite a bit of grey area for statistics, which is why they are exceptionally misleading. What I would call a shot v what you call a shot v what scorekeeper A calls a shot can be very different. Other sports, like baseball and football can and do clearly define whether or not the ball is caught. Hockey has some of the hardest to define statistics among major sports. From a scientific view, the margin for error inherent to statistics in hockey is entirely too large for meaningful analysis. As such, predictions and analysis of games, FROM A PURELY STATISTICAL STANDPOINT, lack an understanding of the nature of the game, and can lead to misleading conclusions.
As a whole, I believe that hockey statistics fail to bring the predictions with the same certainty as with other sports. Part of this is because of their relative youth (A decade simply does not compare to the century Baseball has had) and the other part is the fluidity and somewhat chaotic nature of the game.
I’d also like to point out that some of the harshest critics of the current statistical system I know are in the engineering fields of study, citing them as incomplete, inconsistent and inadequate. Hockey statistics may very well reach the point where they can predict games with an acceptable margin of error, but they aren’t there yet
So if I understand the argument you are making in your first paragraph correctly, you are saying that numbers can’t lie. So by the same argument one could say that words can’t lie. Yet people lie. Numbers themselves can’t lie but the circumstances they are used in can. For example the Apollo 1 mission, the numbers worked but it still exploded because things weren’t taken into account, in base ball statistically the batter should not hit the ball 1/5-1/3 of the time, the number is incredibly smaller than that. And if you disagree that that doesn’t relate to hockey, tell me the stat for trash talk or motivational skills. There are many things in Hockey that can’t be accounted for in statistics, for example a man picks someone else off of a break away so that the other can score.
Now if I understand your argument in the second paragraph, you are saying, one its biased, and two analytics work over time. To the first point, you’re obviously biased too. To the second, I agree that they work over year(s). But the question I then posed, which was the point made in the article, is why is it used in the day to day, because in the day to day statistics don’t decide games.
Now to your third paragraph and stats, one some of the teams stated never won the Cup after 2005-2006 (ex. red wings), which means in data is incorrect, second the data isn’t the same, what counted as a shot then isn’t what counts as a shot now, which means the data isn’t consistent which as anyone can tell you means that the data is invalid because the constant is no longer constant.
For what you said at the end, all I can say is that you need a large pool of data in order get statistics that could possibly be accurate, but the argument in the article is for right now, and right now they don’t have the enough data in order to have a valid argument.
Also several people with doctorates (those polled, had a PhD in either Statistics, or Mathematics),
will be happy to tell you, “Numbers lie, in fact if you don’t check them they will try to lie to you.” Or “Numbers can tell you anything if you present them a specific way.”
“So by the same argument one could say that words can’t lie. Yet people lie. Numbers themselves can’t lie but the circumstances they are used in can. ”
That’s a huge leap to go from numbers don’t lie to words don’t lie, but, ignoring that, I literally said was you said immediately after that – yes, people can manipulate stats to present them in a way they want – but that’s not whats happening, the number go in, the stats come out, and the correlation is made. Nobody is putting these numbers into a taffy stretcher to make them say what they want – in science it is different because people pick and choose which numbers to keep and which to toss to get closer to what they want (or if the data is clearly aberrant and many other reasons that aren’t pertinent to this discussion) whereas with analytics they take the statistical data from the game, put it into the equation and it pops out the numbers, then they see how well it correlates with wins, goals, etc.
“why is it used in the day to day, because in the day to day statistics don’t decide games.”
It’s used day to day to see if teams are trending towards the type of play that the data shows leads to more wins, more goals, etc. Nobody is making the argument that the team with better Corsi in a single game should undoubtedly win the game, they’re saying that teams that are over the course of games taking more shots and allowing less shot are scoring more, and winning more. It’s nothing revolutionary – if I told you there were two teams and one always took more shots and allowed less shots, and the other team did the opposite, you would expect that first team to win more and score more than the second team.
“one some of the teams stated never won the Cup after 2005-2006 (ex. red wings), which means in data is incorrect”
The Red Wings most definitely won the Stanley Cup in 2008, so you should check your own statements before saying mine are incorrect – the winners and the data I presented is accurate.
“which means the data isn’t consistent which as anyone can tell you means that the data is invalid because the constant is no longer constant.”
A shot has always been counted as a shot. They may not have used to count missed shots they way they do now, but they’ve always counted shots on goal and blocked shots as far back as the data goes. Using shots+blocked shots is a stat known as Fenwick which also correlates almost as strongly as Corsi for winning % and 5v5 goal scoring.
Finally, again, numbers don’t lie. People make mistakes, people will fudge numbers, people will make up numbers, people will see things in numbers that aren’t there, but again this is in the context more of science and less in the context of sports stats which are straight plug and chug numbers. How much people want to believe the impact of this correlation has is certainly up to debate – but nobody is suggesting to “watch the game blind” and let the stats tell us everything – you obviously need to watch games because there is a lot going on in every individual game that stats won’t show – what they do show is a real trend of what winning teams do more consistently than teams that aren’t winning.
Nobody who is a big believer in hockey analytics claims that the stats can or ever will be able to predict games – that’s not what they do. As a matter of fact when the organization the NHL partnered with to do their “enhanced stats” said they had an algorithm that could predict playoff teams with 85% accuracy the advanced stats community essentially laughed in their faces because that idea is as ridiculous as it is impossible.
The stats show us TRENDS of what teams that are winning are doing and those trends are there in the data, period.
Regarding the argument that because the shot totals are counted differently – this is true, but is is not some crazy egregious difference. Multiple independent sources that do this for the passion of it (not for money, and there’s no reason for them to fudge numbers) all come up with numbers that are very close – and regardless of whether or not the numbers are exactly the same, they still show the same trends. Even the NHLs crazy system of counting shots shows these trends.
The trends are there and they are not intended to be predictive, they are intended to show what is working for winning teams and this is what it shows.
So, if I understand you correctly, you are saying that the statistics are not used to predict games, but are used to predict trends. I fail to see a reason for tracking statistics if not for predictions or analysis. Said analysis would be used to then make a predictive model, to define a trend, correct? As such, there would be little to no point to simply defining the past based on stats, if said trend would not be carried into defining what makes a good hockey team going forward.
If that is the case, then if I was a coach, I should focus on improving my corsi scores, right? Because all the cup winners have such high scores, that must be what wins games right? If correlation implies causation, anyway.
As such, statistics without analysis for predictive models is meaningless.
But the fact of the matter is that coaches don’t focus on corsi. Its a stat that is fun to play with as analysts, but it isn’t what wins the game at the end of the night. Ask coaches, and their main focus is to score goals. There have been many teams with winning streaks with bad stats, and many teams with good stats and losing streak in the history of hockey, and coaches know that. At the end of the day, goals are what matter. The team with the more wins is the one to look for.
My whole point is that stats don’t show everything. Bu now I ask you how can you count stats to show trends when the stats are created from a limited data pool that isn’t consistent due to human error?