PART 1: Overview, Methods, & top-line results comparing FBS conferences (SEC deep dive), Blue Blood Results, & the state of Florida’s struggles as a whole
(This is the first in a multi-part series comparing each FBS team’s recruiting prowess with actual on-field success)
By Jeff Fuller (@jjfuller72)
College Football is the second most popular sports league in the USA, boasting an estimated 182 million fans and representing a multi-billion dollar industry. Recruiting services help feed fan’s year-round obsession with their teams and have become large businesses themselves. There is strong evidence that the higher rating a recruit receives, the more likely he is to be drafted by an NFL Team and that teams with consistent top-tier recruiting classes not only place the most players into the NFL, but have been the only ones winning national championships (original “Blue-Chip Ratio” article by Bud Elliott in 2014). Many other reports (here, here, here, and here) have also found strong, though not always pervasive, correlations between recruiting prowess and on-field success.
However, players’ increasing use of the transfer portal has made using HS recruiting classes alone for predicting current roster talent less reliable and predictive. Fortunately, the 247 recruiting service has addressed this by producing a Team Talent Composite ranking every year since 2015 that tallies the high school ranking of every player on each team’s current roster, regardless of where they previously signed/played (though many think there should be a “re-evaluation” of team talent based on actual performance after high school, where a HS 2-3 star late-bloomer transfer player has been playing like a 4-5 star talent, a la Cam Ward.)
If the recruiting service rankings only served to pump up preseason magazines, sell subscriptions to insider recruiting sites, or attract the most eyeballs/clicks for selling ads, I don’t think anyone would have that big of an issue whether or not these services “get it right” on their evaluations. But recruiting rankings play a big part in:
- Many CFB computer rankings systems, such as ESPN’s FPI and SP+, rely on these recruiting rankings as a factor for ranking all FBS teams, setting the framework each season on “quality wins/losses” & strength-of-schedule metrics, all of which influence individuals who cast AP/Coaches Poll ballots & the CFP Committee itself.
- With “recruiting momentum” and program reputation always under scrutiny, recruiting classes can snowball in a good or bad way depending on how many highly rated recruits commit.
As stated above, more talented rosters generally translate into greater on-field success, though there are many examples of teams that both under- and over-perform their talent level, not just on a year-to-year basis, but consistently. Josh Pate has recently recognized the need to make a “Military and Mormon exception” to his Power Rating model (which relies heavily on recruiting rankings) for the Service Academies (Army, Navy, Air Force) & BYU.
This begs some questions:
- What factors influence recruiting rankings?
- How often are said rankings reliable in predicting on-field success?
- Are there commonalities in teams that routinely under-/over-perform their recruiting rankings?
- Can we identify any clear patterns of bias or other trends in recruiting rankings themselves?
To delve into this, I looked at the average Team Talent Ranking (TTR) among FBS schools from 2015-24 (247 started measuring this in 2015) and compared that to the Massey Composite Ranking (MCR) over that same ten year period.
Methods: “Team Talent Ranking” (TTR) was measured by recording the 247 Team Talent Composite rank (raw rating score not used due to massive spreads & grade inflation over time) of every FBS team frrom 2015-24 and then averaging those after dropping the highest and lowest years. “Results” were measured by recording the final Massey Composite Ranking (MCR) from 2015-24 (2024 results here) and, similarly, dropping the highest and lowest years. The Massey Composite is derived from averaging 75-90 different ranking systems that all try to identify the best teams every year via their own unique criteria. The 247 Team Talent Composite is calculated by using the average High School rating of 4 different recruiting services (247, On3, ESPN & Rivals) for each player on the roster. The “Talent vs Results” figure was calculated by subtracting the MCR from the TTR. Teams with a positive figure were considered as “overperforming” expectations based on their TTR and those with a negative figure were listed as “underperformers.” Additionally, data for the last five years were separately calculated without dropping any of those years. Some statistical quirks* & examples are discussed at the end.
Key Findings:
-While the SEC had the highest average talent (TTR) and results (MCR), they also came out as the most underperforming conference, with none of their 16 teams having a 10 yr average MCR higher than their average TTR (as discussed below, some of this is a statistical “problem of their own creation” by having such a high baseline of team talent/TTR.) Alabama, Oklahoma, & Georgia performed the best in the SEC at -2, -2, & -4 respectively, while Vanderbilt, South Carolina, and Arkansas pulled up the rear at -36, -34, and -34. SEC teams underperformed their TTR by -17 spots on average. In the last 5 yrs, Ole Miss was the sole overperformer at +6, but the conference as a whole underperformed with an average of -18.
POWER 4 vs Non-P4:
-The Avg P4 team (including Notre Dame) finished 10 spots lower in MCR than TTR, while conversely, the average non-P4 team finished 11 spots higher. Interestingly, the 8 traditional “Blueblood” FBS programs (Alabama, Notre Dame, Ohio St, Michigan, Texas, Oklahoma, USC, & Nebraska) did not show any significant difference in underperformance, at -10, than the average P4 program. If we remove Nebraska from “Blueblood” status (as some have more recently recommended), the average Blueblood underperformance is only -6. In fact, all of the Bluebloods met their recruiting expectation pretty well except for Nebraska (-37), USC (-21), and Texas (-19), an impressive performance given how high of an average TTR they all started with (again, except Nebraska with their avg TTR of 24th, 12 places lower than the next lowest Blueblood’s TTR of Michigan at 12th.)
Conference Results:
-The Avg SEC team finished 17 spots lower in their MCR, at 35th, not nearly matching their nation-best TTR of 18th. The ACC and the Big Ten were next at 12 and 11 spots respectively, while the Big 12 was only at -1. Examining just the last 5 years showed even more pronounced trends with the ACC being nearly as underperforming as the SEC (-17 & -18 respectively) and the Big 12 showing more over performance at +2 than even the American Athletic Conference at 0.
Pre-Realignment Conference Results:
-Some might balk at these relatively historical results due to the inclusion of teams that were not part of their current conferences for more than a year or two. However, prior to recent conference realignment following the 2022 season, the results strongly followed the same trends. The former Pac-12 showed the second most underperformance, at -14, behind the SEC’s -18. While the ACC (-11) and Big 10 (-10) weren’t that far behind, the Big 12 (even with Oklahoma and Texas) showed negligible underperformance at -1; and this is despite the Big12 being the conference with the second best average MCR of 42nd, slightly behind the SEC’s average of 36th. When looking at the different groups of schools who have recently moved into one of the current P4 conferences, all the groups have had an underperformance in the -10 to -17 range, except the first 4 new Big12 schools (BYU/Cincy/UH/UCF) who overperformed as a group at +8 (as would be expected since these schools lacked the “Power Conference” title/“aura”/boost until the last 2 years.) Nearly all these trends were more pronounced over the last 5 years than over the entire 10 year period.
FLORIDA
-In subsequent installments of this series, evidence will be presented showing that teams located in the largest states are usually underperformers while the smallest state schools nearly always overperform. Florida-based schools have massively underperformed their TTR by an average of 18 spots over the last 10 years, and by a whopping 30 spots over just the last 5 years. Even non-P4 teams such as USF, FIU, & FAU seem to either have rosters full of players with inflated talent rankings or all of those “Sunshine State” teams consistently underperform their talent. UCF is the best performing of all Florida-based schools at +8 over 10 yrs and -5 over the last 5, while Florida St was the worst at -35 and -41 respectively.
*Key statistical reminders/quirks:
-The top recruiting teams cannot significantly “over perform” (and the worst recruiting teams can’t really “underperform”) in this type of analysis. For example, Alabama averaged a recruiting class of 1.1 and an avg MCR of 2.6, for a net “underperformance” of -1.5. However, it would have been impossible to “over perform” and get an MCR better than 1.1. Their MCR of 2.6 was actually the best of all 130 FBS teams analyzed, but, due to higher variance in MCR results than TTR, they just couldn’t match their utter dominance in the recruiting rankings. I considered converting the raw averages back to a rank (in this case, Bama would have had a MCR of 1.0) but there were so many irregularities introduced that it made the data less reliable. For example, Iowa had a very respectable MCR of 22, but this was actually 10th best in the nation and most wouldn’t say that Iowa’s results over the past decade were on par with a top 10 team (Iowa’s only finished in the AP top 10 once since 1988, a 9th place finish in 2015, and although they finished ranked 5 of the 9 years since then, it was never above 15th nationally.) Additionally, eleven teams ended with an MCR between 43.8 and 46.8. Re-ranking would have given the best of those an MCR eleven spots better than the one at 46.8, when the average rank was just 3 spots different.
-The Service Academies provide the other side of the story. Since they had the worst (Air Force), 2nd to worst (Army), and 5th worst (Navy) average TTR, it would have been impossible to significantly “underperform.” While all three still far outpaced expected results based on talent level, all finishing with an MCR in the mid sixties, some of their over performance can be attributed to a “statistical advantage” of being at the very bottom in TTR.
-Similarly, if a hypothetical conference had all 16 of the top 16 teams in recruiting rankings, there is no way for them all to end up as the top 16 teams in the MCR. Since those schools all play each other, and every game has a loser, there would be way too many losses taken to finish so high in the MCR. The opposite is true for a hypothetical conference with the worst 16 recruiting teams nationally. While clearly no real-life conference is that strong/weak in recruiting, this reminder adds some context to those with the highest and lowest avg TTRs (The SEC with an avg TTR of 19 & the MAC and MWC-leftovers who had TTRs of 107 & 110 respectively.
