MSU Football Recruiting Retrospective, Part 3: Booms and Busts

In Part One of this series on MSU recruiting, I explored the true value of recruiting stars and introduced a new way to quantify recruiting class quality based on NFL Draft Potential. In Part Two, I broke down MSU's last several recruiting classes and put them into context both in the Big Ten and nationally. We learned that recruiting class ranking is far from linear and recruiting prowess can largely be dividing into four tiers: Tier 1 (Elite/Top 10), Tier 2 (Top 25), Tier 3 (Top 40), and Tier 4 (the rest of the Power Five).  In the Mark Dantonio Era, MSU was largely a Tier 3 program, which is still good enough to be roughly 5th best in the Big Ten behind Ohio State, Michigan, Penn State, and Nebraska.

In the previous two entries of this series, I focused almost exclusively on "input," which came from high school recruiting data. This data was used to make comparisons based on a projection of how much talent is contained in each class and team.  While this data is valid, as averaged over several years and the entire Power Five, each class and team in reality can differ quite a bit in actual on-field productivity.  I will now turn my attention to this reality.

In my original analysis of recruiting data, I focused on a single measure of "output" which was whether or not a player's name was ever called on NFL Draft weekend. The reason for this was largely due to convenience. It is simple, binary (yes or no), and it can easily be applied to a large data set. But, there is certainly a lot more to the value that a player brings to a roster than whether or not they eventually get drafted.

So, as a next step, I decided to dig deeper into teams' rosters to find a way to quantify each player's value after they get onto campus. I settled onto five basic possible outcomes, all of which can be quantified based on a simple set of rules:

Definitions

1. NFL Player

In this case, my definition of NFL player is more broad than just whether or not they were drafted. Several former college players wind up as undrafted free agents and then go onto successful NFL careers. But, it is sometimes difficult to know whether a player true "made it" in the NFL or not.  So, I decided to count a player as an "NFL Player" if they were either drafted (even if they never played a snap) OR if they are listed in the Sports Reference website database as having played on an NFL team for at least one year.  Once again, this provides a consistent set of data that is fairly easy to mine.  However, I should note that more players (the undrafted free agents) fall into this category than they do in the analysis used to develop the NFL Draft Potential metric. This is an important distinction to keep in mind.

2. All-Conference Player

I make no distinction here between a first team player and just an honorable mention player. If a player appeared on either the media or coach's all conference team at some point in their career, then I count it.  

3. Starter

I had to make a judgement call here as to how many starts counts as being a "starter."   Sometimes players are pressed into emergency action for a game or two, or are given a start on Senior Day, for example. So, the threshold to be a starter needs to be greater than one or a few starts.  I settled on over half of a regular season (more than six starts) as the threshold that I will use for this category.

4. Contributor

Sometimes a player does not get many starts, but stays in the program as a quality back-up or special teams player.  If a player earned more than two letters, but less than seven starts, I classify them as a contributor.

5. Bust

If a player fails to earn more than two letters and has fewer than seven starts, they really did not accomplish much while on campus, and I label them as a "bust."  In most cases, these are players that wind up transferring, having a medical situation or injury, getting kicked off the team, or never wind up on campus in the first place.

While there is no perfect system, these criteria can be consistently applied to any team and each recruit can be placed in one of the five clear categories of varying level of contribution.  Unfortunately, collecting the data for each player is labor intensive (I basically had to go through each player's bio on the athletic department websites), and so far I have only gathered a complete set for two teams: MSU and our chief rival, Michigan.  

Comparison Based On Recruiting Ratings

It is easiest to compare the data for the two teams side-by-side. First, let's take a look at the breakdown of "booms and busts" as a function of the recruit Rival Rating / Star bins for the 2007 to 2015 recruiting class (as several members of the 2016 class still remain on each squad as red-shirt seniors).  Also note that I only consider the contributions for players that signed a letter of intent as a scholarship recruit out of high school to either team.  Transfers and walk-ons are not counted in this analysis.

Figure 1: Breakdown of player contribution level as a function of incoming recruit ratings for both MSU and UofM for the 2007 to 2015 recruiting classes

There is a lot of data here and a lot to discuss. First off, it is interesting to simply compare the distributions of players in each recruiting bin for the two schools.  As a general rule, the distribution for UofM is shifted up by approximately one recruiting bin compared to MSU.  The Wolverines roster is mostly stocked with low four-star (5.8) and high three-star (5.7) recruits, while the majority of MSU's roster falls in the three three-star bins (5.5 to 5.7).  Also note that the distribution for MSU is more broad (note the difference in the y-axis).

The Figure also shows the distribution of players in each contribution bin.  Clearly, each recruiting bin has multiple players with dramatically different outcomes.  Based on the different total number of players in each bin, it is tricky to draw many conclusion with the data in this form. For this reason, I find it useful to instead scale the data based on percentages, which is shown in Figure 2

 

Figure 2: Breakdown of player contribution rate as a function of incoming recruit ratings for both MSU and UofM 2007 to 2015 recruiting classes

With this set of data, it is a bit easier to compare both the relative level of contribution of each recruiting bin within each team and between each team.  Let's start at the top with the five-star (6.1) and high four-star (6.0).  As we have learned, these recruits are generally consensus Top 75 players out of high school and there are not than many of them to be had.  MSU has only five recruits in these bins over the nine classes in this analysis.  Michigan has had only 14.  So, the sample size is fairly small, but the data does tell a story.

In this case, the story is significantly more positive for MSU than it is for Michigan. Of MSU's five recruits in these bin, three of them wound up in the NFL (Gholston, McDowell, and Edwin Baker) while one other was an All-Conference Players (L.J. Scott). The only bust in this group for MSU was 2009 offensive lineman David Barrent.  For Michigan, however, the results are not nearly as good. Of the Wolverines 14 players in these bins from 2007 to 2015, only three made the NFL (William Campbell, Kyle Kallis, and Jabrill Peppers) and two were All Conference players (Donovan Warren and Pat Kugler).  Four more players were simply starters, and five were either busts or contributors.  Although the sample size is small, high-level recruits have clearly done better at MSU, on average, than in Ann Arbor over these nine classes.

As we slide down to the mid-four star recruits (5.9) the sample sizes are a bit bigger, and MSU still holds a clear advantage.  The NFL rate for both teams is right around 30 percent, but MSU has a clear advantage here is sending 5.9-rated recruits to All Big Ten teams.  Two-thirds of MSU's mid four-star recruits are either make the NFL or at least make an All Conference team.  For Michigan, that number is only 40 percent.  In addition, the bust rate for 5.9-rated players is very low is East Lansing at only 10 percent.  It is three times higher in Ann Arbor.

As we move down to the mid-level four-star (5.8) recruits, we see the first category where the Wolverines can claim some level of victory, sort of.  Regarding the NFL rate, the Wolverine do well here and have put roughly 30 percent of their recruits in this category into the NFL.  That is a rate that is twice as good as MSU's roughly 15 percent rate.  Considering that the 5.8 bin is the largest one for the Wolverines, this is a significant number. 

However, the results are not all positive. If we instead consider the combination of the NFL and All Conference rate, both teams have a very similar value (37 percent for MSU to 38 percent for UofM).  Furthermore, MSU's bust rate of only about 20 percent is significantly better than the roughly 35 percent bust rate at Michigan. On balance, this recruiting bin is at best a push for Michigan.

Moving on, let's discuss the situation in the three three-star bins together.  For MSU, these three bins, which compromise a majority of MSU's rosters, show a pretty clear trend. At the high end of the range (5.7) MSU sends about 24 percent of all recruits to the NFL and 38 percent to either the NFL or All Conference teams.  The bust rate is equal to the NFL rate (24 percent).  As we slide down to the mid and low parts of the three star spectrum, the performance gets slightly worse. For the 5.6 bin, the NFL rate is down very slightly to 23 percent and the combined NFL / All conference rate is down to 33 percent.  The bust rate is up to 38%.  For the 5.5 bin, the NFL rate is down to 17 percent, the combined NFL / All conference rate is at 28 percent, and the bust rate is high at just over 50 percent.

Once again, it is hard to find any clear advantage in the three-star range for the Wolverines.  In the 5.7 bin, the NFL and combined NFL / All Conference rates are 19 percent and 30 percent, while the bust rate is 38 percent (all worse than MSU).  In the 5.6 bin those numbers are 20 percent (NFL), 24 percent (combined) and 38 percent (bust).  The bust rate is the same as MSU's but the other values are lower.  For the 5.5 bin, the story isn't much better. Those numbers are 10 percent (NFL), 29 percent (combined, which is one percentage point better than MSU), and 38 percent (bust).  For the three star bins as a whole, the only area where UofM has a clear advantage is the bust rate of the low (5.5) three star recruits.  That's it.

In the two-star bins, there is perhaps a little good news for the Wolverines. The sample size is quite small (only ten players total), but two of them did make the NFL (20 percent) and one additional player made an All Conference Team for a combined success rate of 30 percent.  Granted, two of those players were kickers (Gibbons and Wile), but the Wolverines would be wise to accept any positive news at all in this analysis at this point.  

As for MSU, the Spartans have 28 total players in this collection of bins, a full half of which have turned out to have been busts (compared to 40 percent for Michigan).  But, MSU has put five of these players into the NFL (18 percent) and an additional three players to All Conference Teams (11 percent).  Those eight players include first or second round draft picks Le'Veon Bell, Trae Waynes, and Darqueze Dennard and only one kicker (Aaron Bates).  So, while the raw rate is a bit higher for Michigan, I think it is clear the MSU has had more overall success with two-star recruits.  

To summarize what we have learned so far, raw recruiting data suggests that the University of Michigan pulls in more highly rated recruits compared to MSU and the distribution of players in the recruitment bins bears this out. However, when we actually look at various metrics of on-field success, MSU has better success rates in virtually all recruiting bins.  The only real advantage for Michigan is their ability to put low four-start (5.8) recruits into the NFL at a very solid 30 percent rate.  This is significant, as this represents a large number of players (16 total NFL players) but MSU appears to have an advantage in virtually every other parameter analyzed.

What Have You Done For Me Lately?

The analysis above considers the sum total of all recruits from the 2007 to 2015 classes.  However, as I briefly touched on in Part Two of this series, there is evidence to suggest that MSU's overall success (including on the field, on the recruiting trail, and on Draft Day) has taken in hit since the appearance in the Rose Bowl following the 2013 season.  This also corresponds, roughly, to the arrival of Jim Harbaugh in Ann Arbor.  Based on these factors, it also makes sense to look at the same set of data as a function of time / recruiting class, instead of recruiting bin.  The first comparison of these data sets is shown below in Figure 3

Figure 3: Breakdown of player contributions as a function of the incoming recruit class for MSU and UofM from 2007 to now, with players still on the active rosters not counted

While the raw data per player is interesting, once again it is easier to visualize the difference in classes if we convert this data to rates (percentages).  That data is shown in Figure 4

Figure 4: Breakdown of player contribution rate as a function of the incoming recruit class for MSU and UofM from 2007 to now

For these graphs, the trend are a bit harder to interpret, as there is significant class-to-class variation. However, there are several things to note for each team.  Regarding MSU, the data here confirms the comment from Part Two that starting with the 2012 class, the overall output of each class as measured by both NFL-level and All Conference level players, has taken a significant down turn.  Prior to 2012, MSU was placing players in the NFL at a rate of 33 percent overall and into the All Conference Teams at an additional 12 percent rate (45 percent combined)

Starting with the 2012 class, the NFL rate has taken a major hit (below 10 percent) and the combined NFL / All-Conference rate has only been significantly over 20 percent once (35 percent in 2015). The bust rate for MSU has also crept up in the recent few classes.  From 2009 to 2012, the bust rate hovered around 30 percent.  From 2013 to 2015, it was 38 percent total.  While I have not fully counted the 2016 class and beyond yet, that class has been an absolute dumpster fire for the Spartans and current shows a bust rate of 80 percent.  Hopefully, the current coaching staff can start to reverse these trends starting with the 2017 class and those who remain from 2016.

As for Michigan, the trends are a bit more erratic, but seem to be headed in the opposite direction. For the classes between 2007 and 2011, the performance of each class was generally poor (excluding 2009, which had a very high NFL rate of 40 percent).  For the four classes in this range excluding 2009, the NFL rate was only 13% and the combined NFL / All Conference rate was barely over 20 percent. Furthermore, the bust rate for the 2010 and 2011 classes was very high (57 percent in total.)  Those four classes are frighteningly similar to the output from MSU's classes between 2012 and 2016 (so far).

Starting with the 2012 class, Michigan had a strong run of output, with an NFL rate of close to 30 percent overall, and a combined NFL / All Conference rate of 44 percent (very similar to MSU's run from 2007 to 2011, albeit with significantly higher rated recruits).  That said, the classes from 2012 to 2015 show a general downward trend in productivity, with the 2015 class showing only 14 percent rate to both the NFL and All Conference Teams and a high 50 percent bust rate.  As for Michigan's not-quite-complete 2016 class, it appears to be feast or famine, with a 46 percent bust rate yet a 33 percent NFL rate.  As a final note here, 2017 for Michigan is already looking like a problem, as 10 players are already in the bust category.

Add It Up

This is quite a bit of data to absorb, so I will try to summarize my main takeaways:

• Based on recruiting rankings, Michigan brings in about 1.4 more players with NFL Draft Potential per class than MSU (from Part Two)
• MSU's equally-ranked recruits have outperformed Michigan's recruits in almost all categories with respect to NFL, All Conference and Bust rates
• But, the output of MSU's recruits made a significant drop starting with the 2012 class, while in the same time frame, Michigan's recruits have shown improved output, but the trend is a little erratic

In general, the message is a bit mixed. I found it helpful to look at the data from the overall perspective of the 2007 through 2015 classes, as shown in Figure 5:

Figure 5: Overall comparison of MSU and UofM's total batch of recruits between 2007 and 2015, separated by output category (total number of players is shown at the top of each bar)

Interestingly, on balance, MSU and UofM have almost an identical NFL rate in this time frame. The difference is less than a percentage point, with UofM slightly in the lead both in rate and in total players (43 to 40). Michigan's strong performance with low four-star recruits (5.8) is clearly a factor here. Based on the data shown above, it is also clear that MSU had a big lead in this category up until the 2012 recruiting class, and UofM is just now catching up. Based on the results of the 2016 classes so far, UofM will have a lead by this time next year.

However, in the four remaining categories, MSU has a clear lead.  MSU has a better rate of sending players to the All Conference team and has a slightly lower bust rate.  MSU also has a higher rate of starters and contributors, but I do not believe that this is significant. After all, even Rutgers needs to field 22 players and keep 85 players on scholarship.  

As a final note, some might argue that Michigan's higher bust rate (which I believe is referred to as "meritocracy" in Ann Arbor) might be actually more of a positive than a negative. The result has been that over the nine classes in this analysis, the Wolverines have brought in nine more total players (roughly 5 percent more).  More recruits is like getting more at bats: the odds of a home run increase. But, even with more attempts and more higher rated players, MSU still has put more players overall into the NFL and All Conference Teams combined than has Michigan over this time frame.

In conclusion, the results of this analysis essentially tell us exactly what raw recruiting rankings, score boards, and trophy cases have been telling us for a while now.  MSU, as a general rule, has done more with less than UofM over the past decade or so.  But, MSU's has stumbled a bit recently, while Michigan has surged.  Now that the Coach Dantonio Era is officially over, time will only tell us what the future will bring to our intrastate rivalry, both on the field and on the recruiting trail.

While I have a very detailed set of data with which I can compare MSU and Michigan, the same is not true of all Big Ten teams or the other members of the Power Five conferences or the FBS.  That said, in the first two parts of this series, it became clear that the NFL Draft Potential metric can be a powerful tool that can be used to compare teams based on recruiting rankings.  In Part Four of this series, we will discover that it is also a powerful tool that can be used to measure over- and under-achievement, relative to recruiting prowess.  Stay tuned.