The data seem to indicate that recruiting matters and the various services do a decent job of evaluation. It's hard to find a source citing data that really disagrees with that. There are outliers, and I'm routing for the underdog, but in general better rated recruits beat lower rated recruits.
And yes, any given star ranking is anecdotal. The ranking can be derivative, biased, might mess up the evaluation, the recruit might not develop as expected, might fly under the radar, might work exceptionally hard and/or have great coaching and be better than expected, or rankings might just be wrong. We are dealing with individuals, teams, and time. Tons of things matter.
But statistically, the recruit rankings can be used to both pick how well a team performs against another and the odds of a player getting to the NFL --- or even playing in the pro-bowl. Wins and losses as well as playing in the NFL are solid indicators of success for a football player.
NFL Talent on the Roster:
57.6% of 5* recruits were drafted in 2018 (19 players)
23.6% of 4*(70 players)
6.9% of 3* (106 players)
1.1% of 2* (19 players)
?% of NR (42 players)
https://247sports.com/Article/NFL-Draft-...117819292/
By sheer numbers, 3* and below fill the ranks of the NFL. But the odds of any one of those players making it to the NFL are not nearly as high. The more stars a college team is able to recruit, the better the odds they have NFL talent on the roster.
Top level NFL Talent
The 2017 NFL Pro Bowl roster consisted of:
11 college recruits with 5*
24 with 4*
25 with 3*
2 with 2*
13 that were NR
https://www.cbssports.com/college-footba...l-success/
When you run the numbers on how many overall players there are in each category, its amazing how well high-school recruiting rankings predict pro bowl appearances years later. It's great that there are many 3* and below making it too, it helps show that rankings aren't everything, but that's a numbers game (~30 recruits at 5*, thousands at 3* and below, who knows how many thousand if we include NR).
https://www.cbssports.com/college-footba...l-success/
Top Recruiting Classes Play for Top 10 Finishes and Championships
If you look at the top 10 end of season rankings and look back at recent recruiting classes, you'll see a trend, I can't say it any better than the article:
Quote:...of the 32 rosters to compete in the last 16 national title games, 30 of them were formed with at least one top-10 class within the last four seasons. Only Oregon, again, was an exception. But even the Ducks were formed around fringe top-10 classes. 2010 to 2012 Oregon finished 13th, 12th and 13th respectively in the 247Sports Composite Team Recruiting Rankings.
. . .
Of the 70 Top 10 finishers we examined, just 21 percent of teams managed to crack the end-of-year Top 10 without having recruited a five-star prospect within the last four classes. But even those teams had talent. Outside of Boise State in 2011 and UCF in 2013, all of those programs had multiple four-star prospects on their roster as rated by the 247Sports Composite.
https://247sports.com/Article/Why-colleg...114611388/
It's awesome that Boise and UCF crashed the party, but the correlation between recruiting class rankings and top 10 finishes is strong.
Predictive Value: higher ranked recruiting classes usually win
If you didn't know anything else about the teams, the best way to predict the outcome of a game is probably to look at recruiting. You may pick the winning team ~67% of the time. Head to head, or even if you group the teams into classes based on recruiting and average all the results.
Take the P6 teams and make 5 groups, based on the teams' overall recruiting. Then order those groups by the overall wins. For the 4 seasons studied, the results were an amazing correlation of recruit rankings to wins from 5 - 2 (2* and 1* did not appear predictive, I think most would agree that NR - 2* are much harder to predict for a variety of reasons). Those results hold across the tables (not just the overall results).
https://www.footballstudyhall.com/2014/2...ings-right
Like the other metrics, they do not indicate that recruiting is the end-all be-all. There are people that make a hobby of seeing who under or over performs (I always like Bill Snyder). But recruiting ranks seem to have a strong correlation.
http://www.sportsonearth.com/article/108...igning-day
https://fivethirtyeight.com/features/whi...ice-versa/
https://deadspin.com/chart-which-ncaa-fo...1640831522
There are even academic papers exploring the recruiting-to-win question. This one seems to indicate that 45% of the variances in end-of-season total wins could be predicted by recruiting alone:
https://pdfs.semanticscholar.org/6442/7f...35dfdf.pdf
Other variables come into play, which is what keeps me interested and ever-hopeful (stupid Tulsa). But the data suggests that recruiting matters, a lot, and that the rankings do a pretty good job overall. Even if it pains me to say so.