Against All Odds: College Football Season Wrap-Up

It has been a few weeks since LSU slowly choked the will out of Clemson to claim the National Title for the 2019 season.  It has been even longer since MSU beat Wake Forest to claim the coveted Pinstripe Bowl Trophy. As we await the Big Boy "Big Game" this coming Sunday, and thus put football to bed for the next 6 months, I wanted to take one final look at the year and the statistics that were in 2019.

As for MSU, I will be brief.  The Spartans were a narrow favorite against Wake Forest in NYC (-3.5) and wound up winning by 6.  The game unfolded very similarly to some that we had witnessed all year.  There were scary moments (like Wake's opening drive) and some fun moments (like Mike Panasiuk's touchdown and ensuing penalty).  At the end, MSU tried really hard to choke, but eventually the season ended on a positive note, though ever so slightly.

As for next year, who will be the starting QB? Will there be any new faces on the coaching staff? Who will be the breakout young players?  Can MSU once again get over 500?  Your guess is as good as mine for all of those questions.  Fortunately, we will have 6 months to think and talk about that.  For now, let's instead reflect on 2019.

Betting and Metrics Overview

As I did for every week in the regular season, I made a prediction against the spread for all 39 Bowl Games.  My algorithm went 22-17 (56%) against the spread (ATS) to close out one of its best seasons in years.  Overall, I was 414-359 (54%) ATS.  As for my benchmark, ESPN's FPI, their algorithm matched my performance ATS (22-17) in Bowl Season, but overall only went 372-401 (48%) ATS.  Better luck next year, ESPN!

For the first time this year, I made a series of recommend bets ATS most weeks.  Based on my analysis of historical data, I thought that I could achieve a success rate of ~55%.  While this algorithm was silent for the final 2-3 weeks of the season, its performance overall exceeded my expectations.  For the year, the picks from my algorithm went 49-35 (58%), while similar picks using the FPI went 20-16 (57%).  Overall, the combined strategy went 66-49 (57%).  I will count that as a resounding success.  We will see if my hot streak continues in 2020.

Finally, each week I also issued a series of upset picks for the week at hand.  As for Bowl Season, the upset results table is shown below:

Once again, my algorithm finished strong, getting 3 out of the 4 upset picks correct.  The FPI, meanwhile went 3-3.  In total, 11 of the bowl games ended as an upset.  Overall for year, my algorithm went 34-37 (48%) for upset picks, while the FPI went 22-21 (51%).  While the FPI was a little more accurate, it was also considerably less bold its number of picks.

To this point, now that the year is over, I find it useful to visualize the picks made by each algorithm as well as the actual game results to the opening Vegas spread.  I think that this is the best way to truly understand what these numbers mean.  As for the FPI, the correlation between the FPI picks and the opening spread looks like this:

With a R-squared over 0.9 and a slope of 1.01, that is a very good correlation.  Say what you will about the FPI (and I have dogged it a lot over the years) but that is solid performance.  At least, it is a solid performance in predicting what the spread will / should be.  As for my algorithm's overall correlation with Vegas, that chart is shown here using the same scale:

I would categorize this as "not bad."  The R-squared is ~0.75, but the slope seems rather high at 1.23 which suggests that my algorithm is a bit biased towards picking the favorite.  I have observed this before in my data, and I am not entirely sure why that bias exists.  In the off-season, I am thinking about whether I can or should attempt to modify my algorithm to correct for this bias.  That will give me a fun problem to work on over the summer.

But, for me the most interesting data is the correlation between the spread and the actual game results.  For 2019, that data is shown here:

Is this what you expected to see? The scatter in this chart never ceases to surprise me.  The R-squared value is less than 0.3.  People love to use the phrase "Vegas knows."  While that is generally true on average, that actual game results typically vary from the spread by plus or minus 14 points.  So, the phrase should actually be, "Vegas knows... give or take a couple of TDs."  

A final note on this graph is that the slope for 2019 is almost 1.1, which seems high to me.  This suggests that the favorites in general had a good year.  When I checked the raw numbers, the favored teams in 2019 covered 52% of the time and pushed 2% of the time.  That is actually a pretty big deviation from 50%.  I need to dig into the historical numbers in more detail, but I suspect this is unusual.  It is perhaps also why my algorithm (with its bias towards the favorite) did so well this year.  That is a different analysis for a different day.

Bowl Results

As for the overall results of the bowl games, the chart below summarizes the actual result compared to the opening Vegas lines

At a glance, it is pretty easy to point out the teams that had did relatively well in their bowl games (like UNC, San Diego State, Iowa, Washington, Cincinnati, LSU, and Notre Dame) and those teams that did not (like Utah, SMU, Miami, and Auburn).  It will be interesting to see if this positive or negative momentum carries over to the 2020 season.

In addition to the performance of each team, it is also fun to take a look at the performance of each conference as a whole.  I took a look at this in my Bowl Preview, and it is only fair to revisit this now.  The table below shows the actual performance of each conference (not including the National Title Game) compared to the predictions of both algorithms and Vegas.  I highlighted in yellow the system that corrected predicted the record for each conference

As expected, it was a good bowl season for the SEC, who went 7-2 overall (8-2 with the Title Game).  Most of the rest of the conferences checked in right around 500, while the Big 12 limped to the finish line with a 1-5 record overall.  All of these results were within a game of what was projected by Vegas and most of the predictions for the two algorithms.  The only notable difference is that Conference USA had a better run than expected.  However, that was just going 3-5 instead of 1-7.

Comparison to the Preseason Predictions

Back in August of 2019 in this very space, I provided a detailed preseason analysis of the college football season, including a predicted win matrix.  As a final set of analysis of the 2019 season, I think that it is only fair to see how the actual results of the 2019 season actual compared to these predictions.

Regarding MSU, they clearly underachieved.  The preseason rankings put MSU somewhere between #14 and #23 in the country.  My simulations suggested that MSU would most likely win 8 or 9 games.  In reality, MSU finished the regular season at 6-6 and ranked #42 in my final power rankings. So, my prediction was bunk, right?

Well, not so fast my friend.... While my specific prediction for MSU was off, if I consider the mathematical uncertainty of the preseason rankings (which I lean on in my preseason analysis) my overall analysis of the college football landscape was actually correct.  What my most rigorous analysis of MSU's prospects suggested was that MSU would win 8.04 ± 1.72 games.  

By winning only 6 games, MSU's performance was a little more than a standard deviation below the calculated mean.  MSU finished in only the 12th percentile.  But, if I consider my prediction for all 130 teams, roughly 16 teams are going to fall at or below the 12th percentile mark.  In 2019, the actual number was 19, and MSU just happened to be one of them.  All I can say is that sometime you win some and sometimes you lose some.  Once again, college football is unpredictable.

But, while it is unpredictable, the variability can be quantified.  Based on my historical analysis, the preseason rankings are off by an average of about 20 slots per team when compared to the final power rankings.  In 2019, the number was 19.  As for the accuracy of the different preseason magazines, Phil Steele and the FPI did just slightly better than their competition, with an average deviation of 18.2 and 18.3.  S&P+ was in 3rd place at 18.6, while Lindy's and Athlon brought up the rear at 19.3 and 19.5. I am not sure those differences are truly significant, but they are notable.

In order to better visualize the actual results of the 2019 season with the preseason prediction, below I show the plot of the expected win totals for each team at the end of the season (based on their final power rankings) compared to the expected win total in the preseason.  This give a nice visual of the entire 130 team field and the over- and under-achievers are easy to pick out:

In general, the farther from the central red line, the bigger the over and under achievement.  So, teams like Navy, Hawaii, Western Kentucky, and Louisiana were the most impressive, while teams like Toledo, Akron, Miami, and N.C. State were the biggest disappointments (with MSU right there in the mix).

But, the previous figure only shows the expected win total for each team, and not their actual win total.  I actually chose not to plot the actual win totals because, as they are integer values, it makes the chart hard to read.  As I have considered how to evaluate the difference between the preseason expected win predictions and the actual end-year win totals, I believe it is possible to decompose this difference into three separate factors: ability, schedule, and luck.

Teams' actual performance usually differs from preseason expectations due mostly to ability.  A team is usually just better or worse than we thought that they would be.  This is typically captured by a teams change in ranking from the preseason to the post-season, which as stated above, is off by an average of 20 slots.  However, sometimes a team does a little better or worse than expected because their schedule is either easier or harder than expected.  

It is a little tricky to decouple the influence of ability and schedule, but I figured out a way to estimate it by simulating the season for each team using the post-season rankings for each of their opponents but the preseason rankings for the team in question.

As for "luck," this is simply the difference between the post-season expected win total and a team's actual win total. Another way to put this is that it measures a teams ability to win close games.  Good teams (or lucky teams) tend to win more toss-ups than they lose. Another way to think about this is in terms of flipping a fair coin.  If you flip a coin 10 times, you expect to see 5 heads.  If you get 6 head, you are "lucky" by one flip.  Mathematically, the analysis of football luck works the same way.

One could argue that what is described above is not actually "luck," but something else. Teams that win close games are not just lucky.  They execute and make winning plays at the end of games. So, I would also be OK calling this "grit" or "execution."  I should note, however, that in basketball Kenpom calculates the same metric and calls it "luck."  If nothing else, it is a simple, 4-letter word that fits nicely into charts and tables.  So, let's go with that.

The chart below shows the influence of both the schedule and luck on the final win total for all 130 teams relative to their preseason expected win prediction.

For most of teams (MSU included) the effect of luck and schedule did not change the teams win total by much.  But, for a handful of teams, the impact was several games or more.  For an individual team, the breakdown of all three factors (ability, luck, and schedule) can be visualized in a waterfall chart.  As an example of a team that had no schedule advantage, here is the waterfall chart for MSU

In my preseason analysis, the raw preseason data (not including the known uncertainty of the preseason magazines) predicted that MSU would win 9 games.  The fact that MSU only won 6 was due purely to MSU not being as good as expected.  MSU fans can't blame luck or the schedule, which were basically a push.  As for some of the other teams in the chart above, the story is a bit different.

The team that really seemed to be living a charmed life in 2019 was Pitt and their waterfall chart is shown below

Pitt's preseason rank was #55, but I had them ranked #78 in my final power poll.  Yet, in the preseason, Pitt was "expected" to win just under 6 games.  In reality they won 7.  So, what happened?  A combination of luck and a much easier than expected schedule boosted Pitt's win total by almost 3 games.  It will be interesting to see if Pitt gets a little more "love" in the preseason next year. If they do, there is evidence that it isn't warranted.

The team that benefited the most in 2019 from an easier than expected schedule was North Carolina, whose waterfall chart is shown below.  The Heels were only expected to win around 4 games, but the Charmin-soft ACC schedule gave them an additional 2 wins right off the bat.  In addition, UNC was actually better than expected by a game and a half, which should have pushed their win total close to 8.  Alas, UNC was also one of the more unlucky teams in 2019, and their final win total settled at just 6.

As for the luckiest team in 2019, that honor appears to go to Nevada, whose waterfall chart is shown below.  Similar to Pitt, Nevada's preseason ranking (92) was higher than their posts-season ranking (114) which should have pushed their win total down.  However, the Wolfpack somehow managed to pull out four games over apparently superior opponents by a touchdown or less to get to 7 regular season wins.  Also, similar to Pitt, it will be interesting to see if Nevada is ranked a bit higher than they should be in next years preseason polls.

As for teams where the schedule was less kind, Memphis (waterfall chart shown below) is one of the prime examples.  The Tigers obviously had a good season.  But, it seems like it could have been even better, at least on paper.  In the preseason, I projected Memphis to win around 10 games based on only being around the 40th best team in the country.  As it turns out, they were quite a bit better than that (final rank of 20 in my poll) and perhaps should have run the table in the regular season.  But, the AAC was a lot tougher than expected this year and that impacted Memphis' expected win total by almost 2 games.  They appeared to need a little luck to get to 11-1.

Finally, for some teams, nothing goes right.  The poster child for this scenario is 2019 is poor Army, whose waterfall chart is shown below.

In the preseason, it looked like Army might win 10 games with a preseason ranking of 60. Instead, they won only 4 games against FBS opponents.  While the Black Knights final power ranking was lower (79), it was really the schedule and luck that cost Army so many games.  Army played Tulane, Western Kentucky, Air Force, Hawaii, and Navy, all of which were much better that the preseason magazine though that they were.  In addition, Army lost six games by fewer than 10 points.  Don't be surprised if Army is a bit of a surprise in 2020. 

With that, I would like to officially close the book on the 2019 college football season.  Keep an eye out over the summer for my annual math-based football preview, where I plan to do this all over again.  Until then, enjoy, and Go Green.