Avalanche Prospect Watch: Introducing the Prospect Cohort Success Model

Follow me on Twitter if you’re interested in Avalanche-related musings and Matt Duchene advocacy.

Note: Both the structure and content of this article were directly inspired by the work of Josh Weissbock and Money Puck, who were in turn aided by contributors such as Rhys Jessop and Garret Hohl. Much thanks to them for graciously allowing me to make use of both their previous work and the PCS model that they’ve built. I’ve linked to their work under the “Prospect Cohort Success” heading, and I’d highly suggest checking it out.

Let’s say you’re some sort of reckless deviant with an interest in both sports management and glossy photographic paper. If, in pursuit of these interests of yours, you decided to barge into the front office of a competent NHL franchise and proceeded to brazenly thumb through through their calendar, you would likely find no date circled more emphatically than draft day.

Free agency? Not even close. Sure, making a big splash on July 1 is sexy, but a quiet free agency will rarely have a measurably negative impact on a franchise’s future, and might even spare management from the embarrassment of signing Jeff Finger to a four-year, $14 million deal after mistaking him for Kurt Sauer. Conversely, a lackluster day on the draft floor might set your team back years, and a systematic disregard for the draft and its importance will invariably set your team back decades.

In a league that’s both governed by a salary cap and rife with market inefficiencies, any GM worth his salt knows that the opportunity to draft cost-controlled talent is not an opportunity that should be taken lightly. The identification and exploitation of these sorts of inefficiencies has long been a cornerstone of free-market economics, and many front offices are becoming wise to the comparative value of a contributing player who’s still on their ELC.

So, we’re in agreement, the draft is important. This leaves us with the proverbial $925,000 question: How the hell do we figure out who to take?

Evaluating and Translating Performance

Interestingly, player evaluation is still considered to be more art than science. While there are a handful of scouts who have gained a certain notoriety for their ability to discern “an ineluctable truth about a player” (enter Håkan Andersson), the drafting process still often resembles a game of darts on dollar beer night. In the later rounds of the draft, teams regularly select players based on the singular opinion of a scout who might have only seen the prospect in question a handful of times.

To combat this dearth of information, many analysts have attempted to develop data-oriented tools in order to gain a better understanding of which attributes and factors are correlated with the future success of prospects. Don’t be mistaken though, this is not a case of analytics usurping the fabled “eye test”. Rather, it’s a matter of “how do we figure out which players are worth viewing in the first place?”. Live assessment will always have a place in scouting, but NHL front offices have limited resources and if there’s an opportunity to streamline the scouting process, it should of course be considered.

“Identifying future NHLers is critical to building a successful NHL team. However, with a global talent pool that spans dozens of leagues worldwide, drafting is also one of the most challenging aspects of managing an NHL team. In the past, teams have relied heavily on their scouts, hoping to eek out a competitive advantaging by employing those who can see what other scouts miss. Quite a challenge for many scouts that may only be able to watch a prospect a handful of times in a season. While there has been some progress in the past few years with teams incorporating data into their overall decision making, from the outside, the incorporation of data driven decision making in prospect evaluation has been minimal.” – Money Puck

One of the more prominent prospect analysis tools is the constantly evolving set of NHL Equivalency (NHLE) numbers, which have long been popular amongst hockey analytics writers and researchers. NHLEs convert a prospect’s junior hockey points-per-game to the amount they’d be expected to score if they were to transition to the NHL the next season.

However, over the course of their two years of prospect analytics research, Weissbock and Money Puck discovered a number of issues with these NHL Equivalency numbers, which significantly limits their effectiveness. For example:

• NHLEs assume the player in question will make the jump to the NHL, but that is typically not the case. Prospects often play through intermediate leagues, and development and opportunities can shape their future as a player.
• NHLEs are based on a linear regression method which moves all players towards the mean; this rarely happens.
• NHLEs have to be updated as scoring and talent within leagues changes. This means the NHLE translation factors you would use today for the OHL do not necessarily reflect the same league a decade ago.
• The issues in the translation factors are even more exaggerated when looking at leagues that are affected by smaller sample sizes.
• 17-year-olds who play in European professional leagues have different roles on their teams than top 17-year-olds in North America. They are often depth players without special teams time, which lowers their scoring relative to their North American peers. However, the mere ability to play in a pro league is a strong indicator that the player is likely to become a regular NHLer. This will not be reflected in NHLEs, as they are not designed to consider talented under-age players who have smaller roles.
• Lastly, not all leagues have a translation factor. The large majority of ECHLers are unlikely to play in the NHL, but it’s still useful to know their probability of success seeing as some players are able to make the transition.

Prospect Cohort Success

To address these issues, Weissbock and Money Puck have developed a tool for evaluating prospect potential which they’ve dubbed the Prospect Cohort Success (PCS) model. The introduction to the model can be found here and their full methodology can be found here, but I’ll do my best to summarize.

The theory behind PCS is that if you assemble a cohort of the closest comparable peers for any given player, using variables we know to be statistically significant for draft age players in the Canadian Hockey League (age, height, points per game), that cohort peer group can help inform what type of career we can expect the prospect in question to achieve. Currently, future success is defined as having played a minimum of 200 games in the NHL.

For example, a 6’0, 17 year-old forward who scores at a 1 point-per-game rate in the WHL has almost 600 close peers, of which 22% went on to play over 200 games in the NHL. As such, the PCS for this player would be 22%.

In essence, PCS is used to determine how likely a draft selection is to succeed, which is essentially an assessment of how “safe” a draft selection is. Similarly, PCS is often used to gauge how rapidly a prospect is developing and progressing. To paraphrase Stephen Burtch, there may not be a solution to all of the nuance involved, but with a structured set of parameters we can more readily identify what real value is.

Colorado Prospect Rankings: 2015 Initial Report

These prospect rankings are based on a combination of;

a) The likelihood of Player X reaching the 200 game threshold in the NHL, i.e. their PCS%
b) Player X’s potential ceiling, and
c) The direction in which Player X is trending

For example, Joey Hishon currently has a higher PCS% than Nicolas Meloche, and is more likely to reach the 200 game threshold in the NHL as of this writing. However, Meloche is four years younger than Hishon and has been trending upwards, while Hishon has seen his stock fall since he was drafted, and as such I consider them to be comparable prospects.

Both the comparables and PCS% are based on 2014-2015 data. All referenced 2015-2016 season statistics are as of November 1, 2015. iPCS% refers to “Interim Prospect Cohort Success”, and is based on current 2015-2016 data. However, it should go without saying that our 2015-2016 sample is still quite small, and in many cases iPCS% will not yet be a strong indicator of the direction in which a prospect is trending.

1. Mikko Rantanen
2015, 10th Overall
Age: 19
PCS% (n = 4)
iPCS%: N/A

Despite his recent departure to San Antonio, Mikko Rantanen continues to be Colorado’s top prospect. Rantanen has recorded 3 points in as many games during his brief AHL stint, and his play has ranged from impressive to visibly dominant. It would not be entirely surprising to see him recalled to the Avalanche before the season is over.

Rantanen’s combination of size and ability was quite unique for a draft-eligible player in the Liiga, meaning that he only had 4 comparable cohorts. As such, PCS does not yet have a solid grasp on the type of player he will likely become. However, I’m sure most scouts would agree that Rantanen projects solidly as a top-6 winger in the NHL, and it would be surprising if PCS didn’t reaffirm this notion following examination of a measurable sample of AHL play.

Recent Comparables: Olli Jokinen, Jesse Joensuu

2. Chris Bigras
2013, 31st Overall
Age: 20
PCS%: 18.5 (n = 119)
iPCS%: 47.9

The mobile and incisive Bigras has enjoyed immediate success in the AHL, with 8 points to his name in 13 career games, including 4 points in just 6 games this season. Bigras’ smooth transition to professional hockey at such a young age has caused his PCS% to skyrocket, suggesting that an NHL appearance is rapidly become a case of “when” rather than “if”. Unfortunately, Colorado has a glut of bottom-pairing defenders who are signed to one-way deals, and is currently carrying 8 defensemen on their roster. As a result, it would be surprising to see Bigras in Colorado prior to the 2016-2017 season.

Recent Comparables: Carlo Coliacovo, Patrice Brisebois

3. Duncan Siemens
2011, 11th Overall
Age: 22
PCS%: 30.6 (n = 134)
iPCS%: 21.7

Siemens’ slight early season dip in PCS% shouldn’t be too concerning; after two full seasons in the AHL and corresponding PCS%s of 31.9 and 30.6, we have a solid handle of the sort of prospect he is. Defensemen who haven’t transitioned from the AHL to the NHL by the end of their age 22 season do not project well, but Siemens looked quite competent during his short stint with the Avalanche last season, and was certainly no worse than Nathan Guenin or Brad Stuart.

Ultimately, it will be up to GM Joe Sakic and Patrick Roy to decide if Duncan Siemens has a long-term future with the Colorado Avalanche. If not, it would be wise to trade him while he retains some value.

Recent Comparables: Mark Fraser, Brooks Orpik

4. Joey Hishon
2010, 17th Overall
Age: 24
PCS%: 22.6 (n= 226)
iPCS%: 12.4

Joey Hishon is perhaps the most interesting prospect on this list. An obvious remnant of former Colorado head scout Rick Pracey’s era of drafting, Hishon was a spritely but dynamic offensive player before a number of injuries (particularly a concussion) derailed his career. Hishon has only registered 2 assists in 6 games with the Rampage this season, but with Ben Street having been called up to the Avalanche, Hishon has been slotted in as 1C.

Given Colorado’s relative depth at centre and the fact that Hishon doesn’t exactly fit Roy’s blueprint for the Avalanche (big, fast), it would not be surprising to see him traded at some point this season unless he really begins to flourish in the AHL.

Recent Comparables: Keith Aucoin, Raffi Torres

5. Nicolas Meloche
2015, 40th Overall
Age: 18
PCS%: 19.6 (n = 46)
iPCS%: 5.61

Nicolas Meloche is the whole package: a smooth skater with hockey intelligence, a good shot, and a bit of a mean streak to boot. He’s only put up 7 points in 12 games so far this season, but Colorado’s front office and fans are both hopeful that he’ll be able to take a big step forward this season for Baie-Comeau. A full season of point-per-game production or better would certainly be a positive development.

Meloche’s career path projects to be very similar to Chris Bigras’; expect to see him play these next two seasons in Baie-Comeau before transitioning to the AHL.

Recent Comparables: Simon Despres, Francois Beauchemin

6. Borna Rendulic
Undrafted
Age: 23
PCS%: 17.7 (n = 345)
iPCS%: 10.7

One of Sakic’s many European free agent acquisitions, Rendulic started the season in Colorado, but looked quite overwhelmed during a minimal amount of play-time and was demoted to San Antonio. If Rendulic can improve his skating, then it wouldn’t be surprising to see him take a regular shift in Colorado, perhaps even as soon as next season. He’s got a terrific shot, and size to spare.

Recent Comparables: Bryan Bickell, Travis Moen

7. Jean-Christophe Beaudin
2015, 71st Overall
Age: 18
PCS%: 7.99 (n = 363)
iPCS%: 18.3

J-C Beaudin has wasted no time getting out of the gates this season, having registered 23 points in 17 games while seeing his iPCS% jump from 7.99 to 18.3. Beaudin’s Rouyn-Noranda Huskies are, admittedly, quite the juggernaut, but his early exploits are promising given that he’s viewed as more of a developmental project, his draft season having been his first in the QMJHL. While I’m not willing to indict a player on the basis of a small sample of games, Beaudin’s performance has been impressive enough contextually that I’ve leapfrogged him over Conner Bleackley, who has been comparatively poor.

Recent Comparables: Michael Ryder, P-A Parenteau

8. Conner Bleackley
2014, 23rd Overall
Age: 19
PCS%: 15.1 (n = 251)
iPCS%: 5.11

What to say about Conner Bleackley. The former first round selection has had a rough start to the year, having been stripped of his captaincy while tallying 9 points in 13 games. I’m sure many will argue that the captaincy is irrelevant, and that might be true, being stripped of it certainly can’t be viewed as a positive development. Similarly, while recording 9 points in 13 games doesn’t exactly constitute a poor performance, it’s hardly something to write home about for a once heralded prospect in the midst of his draft + 2 season.

Bleackley has yet to sign his ELC, and it’ll be very interesting to watch how the rest of his year progresses as Red Deer looks ahead to the Memorial Cup.

Recent Comparables: Linden Vey, Marek Svatos, Dean McAmmond

9. Mason Geertsen
2013, 93rd Overall
Age: 20
PCS%: 11.3 (n = 204)
iPCS%: 30.2

Mason Geertsen is another player who’s seen his PCS% rapidly inflate on the strength of playing in the AHL as a 20-year-old. Geertsen began to bloom offensively last year with the Vancouver Giants, but he has yet to exhibit the same sort of ability in the AHL. It will likely be several years before we see him challenging for a spot in the Avalanche line-up. For the time being, Geertsen should be focused on adapting to the pace of professional hockey, and making good use of his cannon of a slapshot.

Recent Comparables: Pavel Kubina, Cam Barker

10. Kyle Wood
2014, 83rd Overall
Age: 19
PCS%: 13.7 (n = 124)
iPCS%: N/A

Not much to comment on yet for Kyle Wood, who’s spent the entirety of this young season on the sidelines with a wrist injury. The towering defender registered 40 points in 67 games last season and has an enormous amount of upside (Colorado’s very own Colton Parayko?), but will likely be something of a project.

Recent Comparables: Alexei Semenov, Dan Girardi

Honourable Mentions: J.T. Compher, A.J. Greer, Julien Nantel

Keep an eye out for the Midterm Report, which will be up on BSN sometime around Christmas. As always, find me on Twitter if you’re interested in discussing anything you’ve read here.