by Jeff Fuller (@jjfuller72)
Are College Football Preseason Polls Accurate and Still Worthwhile?
ESPN’s Matt Barrie made headlines at Big XII Media Days by saying: “It’s my biggest pet peeve in CFB. Preseason polls/rankings, they are garbage, they mean nothing. There are teams that are ranked in the top 10, that won’t sniff the top 40 at the end of the season. Early season preception hurts everyone… (except for those it helps he later corrects/clarifies).” ASU’s head coach Kenny Dillingham, gave a more humorous response when recently asked about preseason polls.
Conversely, CBS’s Shehan Jeyarajah argues that preseason polls help highlight smaller brands, doing them more good than harm. Berry Tramel took a more drastic postition (sub required) suggesting that CFB polls have “run their course” and need to be put out to pasture.
The AP poll expanded from 20 to 25 teams in 1989, and, since then, the predictive accuracy of the pre-season rankings has been trending downward. Per “CougarStats” data, the preseason AP poll historically “gets right” about 14-17 of the final top 25, but that figure has dipped down to 11 or less in 3 of the last 4 years (equaling a 4-yr cumulative accuracy figure of only 49%):
It seems pretty clear that increased roster turnovers from NIL and permissive transfer rules has made predicting which teams will “gel” and which will “crash and burn” more difficult.When also considering the significant “anchoring bias” that preseason polls introduce, more and more people are complaining about their continued use.
When looking strictly at programs that have finished ranked more or less often than they started ranked, we see some interesting and repeating historical trends and can identify the teams that AP voters can’t help but fall in love with each and every off-season:
Over every time period analyzed, USC was the the AP’s most overrated team, except the last 5 years, when the Trojans tied for 3rd with Utah at -3 (finishing unranked 3 times more than they were pre-season-ranked), behind both Texas A&M and Wisconsin with -4 each. Notre Dame, Ohio St, and Texas, while historically very overrated, have been trending less so over the last 25 years (and a smaller but similar trend for Washington, UCLA, and Penn St). Alabama and Michigan are the least overrated of the 8 traditional Blueblood programs, teams that the AP voters can usually “love up” with hype and a high pre-season ranking and still trust to finish ranked.
Granted, most of these programs have proven through the years to perform at a high level, garnering respect and “the benefit of the doubt” regarding receiving a high pre-season ranking in the view of many AP voters. But the huge disparity between some big brands and others begs the questions:
- “Why do some programs live up to their preseason rankings more than others?”
- “Why don’t AP voters use a better ‘retrospectoscope’ and realize that, while some teams consistently let them down, others frequently overperform expectations?”
So, who are these teams that AP voters love to hate? That’s probably the wrong word, but who are the teams that are continually overlooked or dismissed and that AP voters miss on year after year?
BYU is the polar opposite of USC, being the most underrated team in every time frame analyzed but one (and finishing tied for 3rd best, behind Northwestern at +4 and Memphis at +3 in that last 10 year span). Despite BYU’s recent and consistent pattern, pundits continue to dismiss the Cougar’s accomplishments as “flukey” with one ESPN writer even going so far as to fabricate two statistics to fit that narrative just this past weekend.
Boise State’s net underrated-ness has all occurred in the last 25 yrs, where they are tied for first at +5 with Cincinnati and BYU. However, the Bronco’s net underrated-ness has trended off to just +2 over the last 10 years and +1 over the last 5.
Northwestern was the most underrated team over the last 10 years, achieving 4 more final than pre-season rankings. Nearly all of this statistical effect came from their consistently over-performing squads between 2015 and 2020.
Utah was among the top 5 most underrated teams in AP poll history until the last decade and the fortunes have flipped with the Utes being tied for the 3rd most overrated team in the past 5 years. While some of this was clearly earned pre-season respect from their back-to-back Pac-12 championships, they also seem to have benefited from “the shine” of P5 status and beating frequently overrated Pac-12 programs (plus unpredictable QB availability).
The analysis above is, admittedly, of limited usefulness due to:
- The binary nature of the results: while it tells us who finished ranked/unranked, it doesn’t tell us how much any given team rose or fell. A team with the top preseason “others receiving votes” position could finish 25th in the final poll and be given as much credit at +1 as a team that received no preseason votes but finished top 5 in the final poll.
- The raw net number doesn’t tell us how many opportunities any given program had to exceed or underperform preseason expectations. For example, Clemson, Iowa, Oregon & USF all had a net “0” figure over the last 50 years, but did so in the setting of 31, 21, 20, and 2 total preseason rankings respectively. While all those programs “met their preseason expectations,” USF had a much lower bar to reach than the others.
- The raw net number can only tell us the total average result over a given time frame, and not how consistent a team actually was from year to year. Hypothetically, a team that had 20 preseason and 20 final rankings over 50 years has a net of “0” but could have gotten that number by finishing ranked every year they started ranked, or could have never finished ranked with a preseason ranking but had 20 other years where they weren’t pre-season ranked, but exceeded expectations and finished ranked.
Fortunately, other analyses help us address most of these limitations. Part 2 of this series discusses two different sets of data (over the last 35 and 50 years respectively) comparing where each team started and/or finished in the rankings
