From SB Nation:

1. Projected S&P+ (5-Year Weighted Average). This is exactly what it says -- a five-year weighted S&P+ average, with 2013 getting 5x the weight, 2012 4x, ..., and 2009 1x. It accounts for about 60 percent of the overall projections. In college football, the best predictor of what you will do is what you've done. Things don't change a whole lot from year to year.

2. Projected S&P+ (Returning Starters). This takes last year's S&P+ ratings and adjusts for your number of returning starters. Obviously returning starters is a pretty vague, debatable number, but at this stage in the game, one month removed from the last season, it's what we have to work with. For now, the best starter numbers to work with are probably Phil Steele's. For my projections, I'll replace his numbers with mine (which are more about top contributors than simply who started) as I derive them. For now, Phil's will do just fine. This number accounts for about 30 percent of the overall projections.

3. Projected S&P+ (2-Year Recruiting). As discussed previously, I have become convinced that a five-year recruiting average is redundant; by the time you're 3-4 years removed from a recruiting class's signing, you've replaced potential (which is what the recruiting rankings reflect) with production. Using two years (as approximately 10 percent of the overall projections) attempts to fill in the gaps between the number of players you lose from last year and the quality of players that will be replacing them.

That's pretty much it. 60-30-10(ish). In Phil Steele parlance, I'm using THREE SETS OF POWER RANKINGS. Below are the projections. (NOTE: By the time the FO Almanac rolls around, we'll have come up with rough projections for the FBS newbies -- Georgia Southern, Appalachian State, Old Dominion -- but for now these are just the FBS teams that were also FBS teams last year.)

The Top 4
1. Alabama
2. Florida State
3. Oregon
4. South Carolina

17. Notre Dame

ACC
14. Clemson
20. Louisville
22. Virginia Tech
36. North Carolina
37. Miami
43. Pittsburgh
44. Georgia Tech
55. Duke
63. Virginia
66. Syracuse
68. North Carolina State
80 Boston College
89. Wake Forest

http://www.footballstudyhall.com/2014/2/...rojections

Syracuse will be better than 66th best team next year

In my opinion, this kind of analysis, while interesting, is somewhat problematic as a predictor of future success. IMO, it adopts a linear construct in using the 5 year and 2 year data, never bothering to factor in random elements such as coaching changes. This is clearly evident in the case of BC. During the 2/5 year window in which this data was used, BC had a disastrous coaching change which drove BC's recruiting and performance to levels far below what they had enjoyed for the previous decade. With another coaching change this past year, BC's performance and recruiting significantly improved. However, this study does not take that into account and only uses data from the nadir of BC's performance/recruiting history.

IMO, for this study to have credibility, the data would need to be run through a sensitivity analysis which takes into account these significant variables, which will influence the results.

Just my opinion and no offense intended to the authors, but this type of analysis is best left to the professionals.

(02-10-2014 05:46 PM)Eagle78 Wrote:  In my opinion, this kind of analysis, while interesting, is somewhat problematic as a predictor of future success. IMO, it adopts a linear construct in using the 5 year and 2 year data, never bothering to factor in random elements such as coaching changes. This is clearly evident in the case of BC. During the 2/5 year window in which this data was used, BC had a disastrous coaching change which drove BC's recruiting and performance to levels far below what they had enjoyed for the previous decade. With another coaching change this past year, BC's performance and recruiting significantly improved. However, this study does not take that into account and only uses data from the nadir of BC's performance/recruiting history.

IMO, for this study to have credibility, the data would need to be run through a sensitivity analysis which takes into account these significant variables, which will influence the results.

Just my opinion and no offense intended to the authors, but this type of analysis is best left to the professionals.

Eagle, you have passed the comprehensive exams and are now cleared to present your defense. :)

(02-10-2014 07:20 PM)lumberpack4 Wrote:  

(02-10-2014 05:46 PM)Eagle78 Wrote:  In my opinion, this kind of analysis, while interesting, is somewhat problematic as a predictor of future success. IMO, it adopts a linear construct in using the 5 year and 2 year data, never bothering to factor in random elements such as coaching changes. This is clearly evident in the case of BC. During the 2/5 year window in which this data was used, BC had a disastrous coaching change which drove BC's recruiting and performance to levels far below what they had enjoyed for the previous decade. With another coaching change this past year, BC's performance and recruiting significantly improved. However, this study does not take that into account and only uses data from the nadir of BC's performance/recruiting history.

IMO, for this study to have credibility, the data would need to be run through a sensitivity analysis which takes into account these significant variables, which will influence the results.

Just my opinion and no offense intended to the authors, but this type of analysis is best left to the professionals.

Eagle, you have passed the comprehensive exams and are now cleared to present your defense. :)

I found it pretty interesting and at least it was attempted to be done in the most unbiased way possible.

(02-10-2014 07:56 PM)Maize Wrote:  

(02-10-2014 07:20 PM)lumberpack4 Wrote:  

(02-10-2014 05:46 PM)Eagle78 Wrote:  In my opinion, this kind of analysis, while interesting, is somewhat problematic as a predictor of future success. IMO, it adopts a linear construct in using the 5 year and 2 year data, never bothering to factor in random elements such as coaching changes. This is clearly evident in the case of BC. During the 2/5 year window in which this data was used, BC had a disastrous coaching change which drove BC's recruiting and performance to levels far below what they had enjoyed for the previous decade. With another coaching change this past year, BC's performance and recruiting significantly improved. However, this study does not take that into account and only uses data from the nadir of BC's performance/recruiting history.

IMO, for this study to have credibility, the data would need to be run through a sensitivity analysis which takes into account these significant variables, which will influence the results.

Just my opinion and no offense intended to the authors, but this type of analysis is best left to the professionals.

Eagle, you have passed the comprehensive exams and are now cleared to present your defense. :)

I found it pretty interesting and at least it was attempted to be done in the most unbiased way possible.

I agree it's an attempt to be unbiased, but the use of whole class rankings bothers me. Having a top B-5 ready QB is absolutely crucial to success and not having one can be lost in the ranking of an entire class. To give an example, NC State went from a Profession Grade QB - Mike Glennon and 7-5 results to no B-5 ready QB at all and dropped to 3-9. Had we a decent QB, we were 6-6 at worst last year.

These rankings don't deal specifically with the QB and I think they have to do that. A coaching change is also hard to control for as most incoming coaches are coming in to replace someone who was not getting it done or who had run the program into the ground. In some cases, there is enough talent in place that a good coach will immediately get results - see Auburn this year and I think Texas next year.

But I am very picky about stats as I once had a very bad experience with them during a dissertation.

(02-10-2014 05:46 PM)Eagle78 Wrote:  In my opinion, this kind of analysis, while interesting, is somewhat problematic as a predictor of future success. IMO, it adopts a linear construct in using the 5 year and 2 year data, never bothering to factor in random elements such as coaching changes. This is clearly evident in the case of BC. During the 2/5 year window in which this data was used, BC had a disastrous coaching change which drove BC's recruiting and performance to levels far below what they had enjoyed for the previous decade. With another coaching change this past year, BC's performance and recruiting significantly improved. However, this study does not take that into account and only uses data from the nadir of BC's performance/recruiting history.

IMO, for this study to have credibility, the data would need to be run through a sensitivity analysis which takes into account these significant variables, which will influence the results.

Just my opinion and no offense intended to the authors, but this type of analysis is best left to the professionals.

AGREED. The only thing this kind of analysis can even hope to tell is how a team will do over the long haul. To do that they would need more like 40 years of data - and it would not tell anything about THIS year, but maybe about the average over the next decade (IF they also took long term trends into account).

(02-10-2014 07:56 PM)Maize Wrote:  

(02-10-2014 07:20 PM)lumberpack4 Wrote:  

(02-10-2014 05:46 PM)Eagle78 Wrote:  In my opinion, this kind of analysis, while interesting, is somewhat problematic as a predictor of future success. IMO, it adopts a linear construct in using the 5 year and 2 year data, never bothering to factor in random elements such as coaching changes. This is clearly evident in the case of BC. During the 2/5 year window in which this data was used, BC had a disastrous coaching change which drove BC's recruiting and performance to levels far below what they had enjoyed for the previous decade. With another coaching change this past year, BC's performance and recruiting significantly improved. However, this study does not take that into account and only uses data from the nadir of BC's performance/recruiting history.

IMO, for this study to have credibility, the data would need to be run through a sensitivity analysis which takes into account these significant variables, which will influence the results.

Just my opinion and no offense intended to the authors, but this type of analysis is best left to the professionals.

Eagle, you have passed the comprehensive exams and are now cleared to present your defense. :)

I found it pretty interesting and at least it was attempted to be done in the most unbiased way possible.

With all due respect, while I have no way of knowing the intent of the authors of this analysis, IMO, it nevertheless is biased in terms of what it purports to be used for - specifically, a predictor of future success.

The data is not biased. Data, by itself, is just that - data. It is the application of this data and the methodology used to form a conclusion that, IMO, is biased. Again, I am not saying that it was the intent of the authors to devise a biased analysis. It is, however, IMO, a biased result based on how the data was used.

If the purpose of this analysis was to simply rank performance over the past 5 years by just doing what it did - e.g., regressing several data points in a basic manner to drive a ranking, I would have less of an issue with this. However, as someone who has done this type of analysis numerous times, IMO, you cannot simply take this analysis to reliably predict future results without either: (a) subjecting it to a sensitivity analysis to take into account the elements that I mentioned in my above post or (b) assigning to the analysis a margin of error which takes into account the issues I have raised.

IMO, there certainly where modifications to the methodologies used which could have been adopted so I would disagree with your comment that this was "done in the most unbiased way possible."

Full disclosure, this is a pet peeve of mine. We see this type of thing occur all to often. IMO, governments and organizations all to often make critical policy and business decisions based on analysis which, IMO, can often have the same kinds of built-in biases (either intended or unintended). That's bad for all of us, IMO.

By limiting the scope of the data analyzed to just the last 5 years, the author is making the assumption that anything that happened prior to that is irrelevant and whatever has occurred in the last 5 years will go on forever... now for, say, a SC Gamecocks fan, they would LOVE to think that because the Gamecocks have enjoyed unprecedented success over the last 5 years. In fact, if you told me this analysis was done by a Gamecocks fan I would not be shocked - after all, it puts them into the 4-team playoff, despite the fact that S Carolina has never finished in the top 4 as far as I can recall...

(02-11-2014 01:32 PM)Hokie Mark Wrote:  By limiting the scope of the data analyzed to just the last 5 years, the author is making the assumption that anything that happened prior to that is irrelevant and whatever has occurred in the last 5 years will go on forever... now for, say, a SC Gamecocks fan, they would LOVE to think that because the Gamecocks have enjoyed unprecedented success over the last 5 years. In fact, if you told me this analysis was done by a Gamecocks fan I would not be shocked - after all, it puts them into the 4-team playoff, despite the fact that S Carolina has never finished in the top 4 as far as I can recall...

You are correct, Mark. Put another way, the smaller the sample size (in this case, 5 years), the greater the responsibility, IMO, to subject the analysis to more sensitivity to take into account events, actions, and trends that do not necessarily get factored into such a small sample size. The greater the sample size, IMO, there is a corresponding less of a need to do this as there is a greater likelihood that these events, actions, and trends are built into the data.