2023 Preview, Part 1: How Accurate are Preseason Rankings?

It has been almost six months since the Georgia Bulldogs claimed the National Title by beating Texas Christian and it has been slightly less than three months since the Michigan State Spring Football Kickoff event. But now that the calendar has turned to July, we are less than two months away from the return of college football.

The Spartans' roster experienced an unexpected shake up in the spring with former offensive stars Payton Thorne and Keon Coleman deciding to leave East Lansing via the transfer portal. Other teams across the country have experienced similar changes. Now that the dust has settled, it is time to start looking forward to the 2023 season.

While there is rampant speculation every year about the next season as soon as the clock hits all zeros in the National Title game, early July is the time when the official preseason rankings start to appear online and on the magazine stands. 

This is also the time of year when regular season win totals are available for betting purposed (e.g. "over/under" bets). I have historically used these data to perform a rigorous analysis of the coming college football season. That analysis starts now for the 2023 season.

I would like to start with some background information that will provide context to some of the simulation results and analyses that will come over the next several weeks. 

Today, I would like to explore the subject of the preseason rankings and the win total predictions themselves. Specifically, how accurate are these predictions?

Comparing the preseason rankings

A full list of college football preseason rankings (of all 133 FBS level teams) can be found in a few different places. In you actually visit the magazine section of brick-and-mortar bookstore, at least three different publications have preseason issues with full rankings: Athlon Sports, Lindy's Sports, and Phil Steele are the publishers that I have utilized over the years for my analysis.

More recently, online sources such as ESPN's Football Power Index (FPI) and Bill Connelly's' S&P+ system both provide a full list of preseason rankings as well. I have historically utilized some combination of all five sources as an input to my college football preseason model.

As a result, I have several years of preseason data which can be easily compared to postseason rankings of all 130-plus teams (based on my model's final power rankings for each year) to determine how accurate the preseason rankings actually are. Figure 1 below provides this comparison for the past six years (excluding the COVID-shortened year in 2020, which is an outlier for a lot of reasons).

Figure 1: Standard deviation of the accuracy of four major preseason rankings services over the past six seasons, excluding 2020.

In this case I calculated the standard deviations between the preseason and postseason rankings for four of the most common ranking services for all 130-plus FBS teams. Note that I excluded Lindy's rankings from this analysis do to an incomplete data set. That said, the trend for Lindy's is similar.

The first thing to note about Figure 1 is that over the six year period these deviations range from 22 to 30 for all four services with some year-to-year variation. The average of the deviations suggestion that  any given preseason ranking is typically only accurate with plus-or-minus about 25 slots.

Second, of the four services shown, Phil Steel is the most accurate overall. This is a claim that Steele makes every year on the cover of his magazine. However, his advantage is honestly very small. Steele's average deviation over this six-year period is 24.8, while the other services are just above 25.

So, while Steele is nominally the best source, all of the preseason magazines have very similar results. I typically use an average of four or five sources each year in an attempt to weed out any outliers.

Accuracy the Preseason Top 25

The fact that the preseason ranking are only accurate by roughly 25 spots helpful, but it is also best applied to the most average teams in the FBS. A team that is ranked No. 40 in the preseason has about a two-thirds chance of finishing the season ranking between No. 15 and No. 65.

However, the situation is a little different for the teams at the top of the rankings. A team that is ranked No. 4 in the preseason, for example, can only move up three spots, but they can drop over 100 spots. 

Figure 2 below summarizes what happens to teams ranked in the preseason top five, ten, and 25 over the course of the season. In this case, I only used Phil Steele's preseason rankings, but I used data for the last 15 years to expand the data set.

Figure 2: Distribution of the postseason rank of FBS teams relative to the teams' preseason rank (from Phil Steele) from 2007 to 2022.

For teams in Steele's preseason top five, almost exactly half of those teams end up in the top five at the end of the season. For the remaining half of the preseason top five, about 19% finish in the top 10, 24% finish out side of the top 10, but still in the top 25, and the remaining 8% finish unranked. 

These data imply that about every other year, a preseason top five team finishes the season unranked. The most recent example that I see is the 2018 Wisconsin team, which entered the season ranked No. 4, but finished the regular season unranked with a record of  7-5.

For teams ranked between No. 6 and No. 10, 19% hold position while an equal number move up into the top five. 33% of these teams drop in the polls, yet remain ranked. The remaining 29% drop out of the top 25. 

For the teams in the bottom 40% of the preseason top 25, 35% maintain their position. 21% of these teams wind up in the top 10 with 7% total rising to the top five. On the other hand, 44% of the teams ranked between No. 11 and No. 25 in the preseason end the season unranked.

Finally, for the much larger group of teams who are unranked in the preseason, the majority (92%) remainn unranked. However, I count eight teams total since 2007 who started the season outside of Steele's top 25 and finished in my top five. I count another 16 teams in that span who finished ranked between No. 6 and No. 10. 

The most notable example is TCU from last year, who Phil Steele ranked No. 51 in his preview magazine. The Horned Frogs finished the regular season undefeated and advanced to the Championship game.

That all said, all but four of the last 15 National Champions have started the season ranked No. 4 or better by Steele. The four exceptions are Clemson in 2016 (No. 5), Ohio State in 2014 (No. 7), Alabama in 2009 (No. 7), and Auburn in 2010 (No. 22).

Win Total (over/under) retrospective

An additional source of information about the impending college football season can be found by looking at the regular season win total (over/under) bets that are starting to be posted by various casinos now. Similar to the preseason rankings, it is valuable to know how accurate these predictions tend to be.

I have been tracking preseason win total bets since 2015. Figure 3 below summarizes the deviation between present over/unders and the actual number of regular season wins for all FBS teams in that span (a total of over 900 data points).

Figure 3: Comparison of the preseason win totals (O/U) and the final regular season win total for all FBS teams from 2015 to 2022 (not including the 2020 season).

Interestingly, the distribution forms a bell curve-like shape which is centered at zero. As it turns out those folks in Vegas that set these lines may know a thing or two. The shape of the distribution is a bit strange, but this is due to the fact that there are not the same number of win total line that are integers as opposed to fractions (for example 5.0 versus 5.5 wins).

The standard deviation of this data set is 2.13, which is the parameter used to generate the "best fit" bell curve line shown in Figure 3. The data in Figure 3 shows that a little under a half (43%) of all win total bets in this timeframe wind up being within a game of the real win total. As expected mathematically, roughly two-thirds (70.5%) of the predictions are accurate to within two games (one standard deviation).

The other implication is that almost 30% of teams (roughly 40 total) do not finish within two games in either direction of their preseason win totals.

Now that this background is in place, over the coming days and weeks I will next provides a brief summary of last year's season. After that, I will turn my attention to the upcoming college football season, including some math and simulation-based predictions and an analysis of some of the preseason betting lines. Stay tuned.