Efficient recruiting spending relative to team performance continues to captivate our imagination, so we examined seven years of NCAA hockey data and NHL drafts spanning 700+ college players (including 350 top prospects) to explore the relationship between NCAA champions in hockey and top prospects. 1 In this study, we define a “top prospect” as one of the top 50 recruits each year according to www.insidecollegehockey.com (an affiliate of ESPN). Team rosters were sourced from www.collegehockeystats.net, which has the largest amount of readily available rosters for college hockey dating back to the 1999-2000 season.
Our study results show a higher amount of top prospects and draftees amongst the teams that were successful in the NCAA Tournament [fig. 1]. The correlation between success in the playoffs and top prospects and draftees was virtually identical at .302 and .303, respectively. The fact that these correlations are so close gives credibility to the study, since both Inside College Hockey and NHL recruiters consider the same players to be the best in the country. While correlations this low are typically deemed far too low to be of statistical significance and are initially unnerving to statisticians, when taking into account that it is a single variable being used as a predictor, it is actually quite relevant. These correlations tell us that distance in the playoffs and these two variables move in lockstep more than 30% of the time, which means that with just one of these variables we can have a pretty good chance of determining which teams will experience success in the NCAA Tournament. Our chances of correctly predicting winners would increase even more were we to incorporate additional variables into this model.
Fig.1 — Top Prospects and NHLers, 2005-2011
There are two interesting things to consider when looking at these data:
1. Teams that won the Frozen Four were not the squads that averaged the most prospects or drafted players, but instead were the runners-up. While the difference is virtually negligible, it highlights the fact that the best team, or the one with the most talent according to our metrics, does not always take away the first prize. 2 In fact, as Baseball Prospectus writers Dayn Perry and Nate Silver noted in their 2007 book “Baseball Between The Numbers,” even the most successful teams in the MLB regular season will rarely have better than a 30% chance of winning the World Series. As applied to our analysis of college hockey, we observe that a team might be among the elite during the regular season and have the best players, but have even less than a 30% chance of winning the trophy because of the one-and-done playoff format that is unforgiving of misfortunes or an off game. This was certainly the case for the 2008-2009 North Dakota team that was extremely talented (13 prospects and 15 draftees) and one of the best teams in terms of record during the regular season, but which lost in the first round of the tournament.
2. Note the steep drop-off in both top prospects and draftees that can be seen between teams that made the Frozen Four and those that lost beforehand. This seems to indicate that while a team of lesser caliber might upset a seemingly superior squad (e.g., the 2008-2009 North Dakota team), the best teams have the tendency to rise to the top as the tournament progresses. Also, there are occasionally teams that are either in a weak conference or play beyond their means in the conference tournament. Because an automatic berth goes only to the conference champions, these squads thus make the NCAA Tournament in favor of another team that might be stronger overall.
The implications of this study might help college hockey departments optimize their recruiting efforts. For teams that struggle to make the NCAA Tournament, we can say with reasonable confidence that having close to four top prospects on a team gives it a pretty good chance to take that next step. Moreover, it seems that the one-and- done system of the tournament gives a team of four leading prospects an upper hand not only to make the playoffs, but also to advance to the Elite Eight. For those teams that consistently compete at a high level in college hockey, we can infer that more than five high-level prospects on a team are typically required to make the Frozen Four, and close to seven prospects on a team are needed to win the championship. The net takeaway: if possible, optimize recruiting efforts to have between four and seven top prospects on the team each season. 3
Observing these data might raise the question as to whether there are enough top prospects to go around for each team to have its fair share. The obvious quick answer would be “no,” since we know before each season that there are teams amongst the 64 of college hockey that do not have a feasible shot at winning the championship. But just how many do have a chance? Over the seven years of this study, there were 29 teams that had at least seven prospects on their squads [fig. 2].
Fig.1 — Top Prospects and NHLers, 2005-2011
Of these 29 teams, three won the NCAA Tournament (2008 Boston College, 2009 Boston University, 2010 Boston College) and four were the runnersup (2006 Boston College, 2007 Boston College, 2010 Wisconsin, 2011 Michigan). Considering that only 14 teams possibly could meet these criteria, it is significant that half of the group is made up of teams that had at least seven elite prospects. If we go on to include teams with six top recruits (since after all, seven is only the average), we find only one more unique team entering the field of having seven or more prospects (New Hampshire), while acquiring a new champion (2006 Wisconsin) as well as a new runner-up (2008 Notre Dame). Thus, it would be fair to say that these 10 teams [fig. 2] are the teams that have a legitimate chance to win the championship year in and year out, and that in any given year one of these teams will make it to the championship 50% of the time.
However, this should not be a major concern to college hockey departments that are not included [fig. 2]. Digging deeper into the data, we find that there are more than enough recruits to go around for each team to have the chance to experience success in the tournament. Of the 350 top prospects considered over the seven years, only 245 made an appearance in the NCAA Tournament. This means that 30% of the most talented players entering college hockey never get the chance to compete in the playoffs! With the understanding that it takes an average of only four players to reach the Elite Eight, teams that have a few high-level prospects would do well to identify players that are going to teams where they will not make the tournament, and show them that their school could offer the chance to skate for a national championship. It is also important to keep in perspective that it is not necessary to have exactly four or seven prospects, since these are merely averages. While there are teams that will have more than average, there are also squads experiencing success that have fewer than the average number of elite prospects expected to be necessary to reach the respective levels of success.
As with any model, this study is not without its fair share of assumptions and limitations. We note three:
1. The model for the study considers only seven years of data, which is limited, especially when considering the teams that went deep into the playoffs. While there are 56 teams in the “tournament” bucket and 28 in the “Elite Eight,” only seven teams’ data are available for the champions and runners-up, respectively. 4
2. The model does not account for how highly touted the prospect is within the rankings system. In other words, two teams that have four top prospects each could be at very different competencies, as one team’s top prospects might be in the top 15 of their class while the other team’s top prospects barely make it into the top 50. Another similar possibility is that certain positions are more valuable, and thus it is more important to have excellent prospects manning those slots; an obvious example being that the best goalies are more valuable than skaters.
3. The model does not consider the tenure of the players with the college team or the playoff experience they have. It is possible prospects that are upperclassmen fare better than freshmen in the NCAA Tournament, just as there is a chance that those who have been in the tournament before fare better as well.
Despite its limitations, this study should still prove useful to college hockey recruiting departments because of the implications outlined above. The clear correlations between draftees, prospects, and success in the NCAA Tournament give credibility to the research, and the steep drop-off at the Frozen Four gives a good idea of the type of talent that is required to reach each phase of the NCAA Tournament. Additional research about the assumptions and limitations listed above would improve the takeaways available in investigative efforts such as this, and might help identify even more ways that recruiting resources could be better allocated.
Key Insights
History shows us that having the most-talented team doesn’t always guarantee an NCAA hockey championship. However, our research indicates that teams with between four and seven “top prospects” are more likely to enjoy post-season success.
References:
- Here’s the line of thought that sparked our interest. When one looks at the players that have been on a team that won the NCAA Frozen Four, many names such as Ken Dryden and Adam Oates stand out as players that went on to have exceedingly successful careers in the NHL. Many other NHL greats played college hockey, if only for a brief time in their careers, and were unable to win the elusive championship trophy. Nevertheless, we would expect that teams that won the championship, were in the Frozen Four, or made the NCAA Tournament were teams that had a lot of players that went on to play in the NHL. We would therefore also expect these players to be highly touted as they entered the college hockey scene as top prospects, so long as the methods of evaluating their talents were efficient. Thus, in looking at these teams, do we find many players that not only were considered elite players as they were beginning their college hockey careers, but also got drafted by an NHL team? ↩
- This is due to the law of large numbers: the elementary theorem of statistics that claims that as a sample size increases, the actual outcome will be closer to the expected value of the experiment. ↩
- Practically, perhaps this means that instead of allocating recruiting resources over many top-level recruits, a hockey department might be best suited to allocate a higher percentage of its efforts to a smaller amount of players. If the team is extensively pursuing far more than seven top prospects, it might be wiser to focus more efforts on players who are geographically closer to the program or those that the program deems to be the most likely to want to join the club, and less effort on players that would increase the number of elite prospects far above seven. This is not to say that having more than the average amount of excellent recruits is not desirable, since naturally there are teams above and below the average and it never hurts to recruit as many of the best players as possible. ↩
- Two solutions to this would be either to use NHL draftees to a greater degree or find another reliable prospect rating system dating further back, or to reevaluate this study in the coming years as more data become available. ↩
