Hello There, Guest! (LoginRegister)

Post Reply 
The Perfect Computer Ranking?
Author Message
quo vadis Offline
Legend
*

Posts: 50,012
Joined: Aug 2008
Reputation: 2372
I Root For: USF/Georgetown
Location: New Orleans
Post: #21
RE: The Perfect Computer Ranking?
(06-26-2020 08:31 PM)Hokie Mark Wrote:  
(06-26-2020 11:09 AM)ken d Wrote:  
(06-25-2020 06:46 AM)schmolik Wrote:  In the old BCS era, I hated the computer rankings. I couldn't understand them. I'd rather use the SOS. Teams are evaluated on known criteria. Most of the computer formulas are so difficult to understand (assuming the formulas are even public and some aren't and if they aren't it only introduces the possibility of bias). I hate it that the NCAA men's basketball went to the NET instead of the RPI. The RPI formula is much simpler to understand and calculate. Does anyone even know how to calculate the NET?

How does making a formula public eliminate the possibility of bias? Every INCORRECT formula is biased. There is no way to know which football team is "best", or whether a team ranked #5 is "better" than one ranked #8 or #9. The simple fact is that the better football team does not always win, so plugging results of individual games into a formula is never going to be better than an eyeball test, or "beauty contest".

The eye test is inherently biased, but mathematical formulas don't have to be.

The bias in a computer formula is in the choices the programmer makes in terms of what factors to include in the formula, how to weight them, etc. And those choices reflect his subjective beliefs about what makes a team good. The biases are not skewed for or against particular schools or conferences, but pertain to abstract team profiles.

That's why it's best to use composites of multiple computers, so that different biases in different formulas might mitigate each other.
(This post was last modified: 06-26-2020 09:14 PM by quo vadis.)
06-26-2020 09:12 PM
Find all posts by this user Quote this message in a reply
bullet Offline
Legend
*

Posts: 66,299
Joined: Apr 2012
Reputation: 3285
I Root For: Texas, UK, UGA
Location:
Post: #22
RE: The Perfect Computer Ranking?
(06-26-2020 09:12 PM)quo vadis Wrote:  
(06-26-2020 08:31 PM)Hokie Mark Wrote:  
(06-26-2020 11:09 AM)ken d Wrote:  
(06-25-2020 06:46 AM)schmolik Wrote:  In the old BCS era, I hated the computer rankings. I couldn't understand them. I'd rather use the SOS. Teams are evaluated on known criteria. Most of the computer formulas are so difficult to understand (assuming the formulas are even public and some aren't and if they aren't it only introduces the possibility of bias). I hate it that the NCAA men's basketball went to the NET instead of the RPI. The RPI formula is much simpler to understand and calculate. Does anyone even know how to calculate the NET?

How does making a formula public eliminate the possibility of bias? Every INCORRECT formula is biased. There is no way to know which football team is "best", or whether a team ranked #5 is "better" than one ranked #8 or #9. The simple fact is that the better football team does not always win, so plugging results of individual games into a formula is never going to be better than an eyeball test, or "beauty contest".

The eye test is inherently biased, but mathematical formulas don't have to be.

The bias in a computer formula is in the choices the programmer makes in terms of what factors to include in the formula, how to weight them, etc. And those choices reflect his subjective beliefs about what makes a team good. The biases are not skewed for or against particular schools or conferences, but pertain to abstract team profiles.

That's why it's best to use composites of multiple computers, so that different biases in different formulas might mitigate each other.

But it still doesn't compensate for the small sample size and limited comparative schedules (most games are within a conference). For the Big 12, they only have 30 games out of conference and usually 8-9 of those are against FCS teams.
06-26-2020 09:30 PM
Find all posts by this user Quote this message in a reply
quo vadis Offline
Legend
*

Posts: 50,012
Joined: Aug 2008
Reputation: 2372
I Root For: USF/Georgetown
Location: New Orleans
Post: #23
RE: The Perfect Computer Ranking?
(06-26-2020 09:30 PM)bullet Wrote:  
(06-26-2020 09:12 PM)quo vadis Wrote:  
(06-26-2020 08:31 PM)Hokie Mark Wrote:  
(06-26-2020 11:09 AM)ken d Wrote:  
(06-25-2020 06:46 AM)schmolik Wrote:  In the old BCS era, I hated the computer rankings. I couldn't understand them. I'd rather use the SOS. Teams are evaluated on known criteria. Most of the computer formulas are so difficult to understand (assuming the formulas are even public and some aren't and if they aren't it only introduces the possibility of bias). I hate it that the NCAA men's basketball went to the NET instead of the RPI. The RPI formula is much simpler to understand and calculate. Does anyone even know how to calculate the NET?

How does making a formula public eliminate the possibility of bias? Every INCORRECT formula is biased. There is no way to know which football team is "best", or whether a team ranked #5 is "better" than one ranked #8 or #9. The simple fact is that the better football team does not always win, so plugging results of individual games into a formula is never going to be better than an eyeball test, or "beauty contest".

The eye test is inherently biased, but mathematical formulas don't have to be.

The bias in a computer formula is in the choices the programmer makes in terms of what factors to include in the formula, how to weight them, etc. And those choices reflect his subjective beliefs about what makes a team good. The biases are not skewed for or against particular schools or conferences, but pertain to abstract team profiles.

That's why it's best to use composites of multiple computers, so that different biases in different formulas might mitigate each other.

But it still doesn't compensate for the small sample size and limited comparative schedules (most games are within a conference). For the Big 12, they only have 30 games out of conference and usually 8-9 of those are against FCS teams.

I've seen some researchers work wonders with sample sizes of 30. True, we don't get the same richness of data that we do with say hoops, but we do get enough to make good determinations. I mean, look at the computers, and then look at who we all know was good last year. There's a pretty strong correlation, unless you really do think that we don't know that Ohio State was better than Memphis and Memphis was better than USF.
(This post was last modified: 06-27-2020 08:38 AM by quo vadis.)
06-27-2020 08:36 AM
Find all posts by this user Quote this message in a reply
ken d Offline
Hall of Famer
*

Posts: 17,335
Joined: Dec 2013
Reputation: 1211
I Root For: college sports
Location: Raleigh
Post: #24
RE: The Perfect Computer Ranking?
(06-27-2020 08:36 AM)quo vadis Wrote:  
(06-26-2020 09:30 PM)bullet Wrote:  
(06-26-2020 09:12 PM)quo vadis Wrote:  
(06-26-2020 08:31 PM)Hokie Mark Wrote:  
(06-26-2020 11:09 AM)ken d Wrote:  How does making a formula public eliminate the possibility of bias? Every INCORRECT formula is biased. There is no way to know which football team is "best", or whether a team ranked #5 is "better" than one ranked #8 or #9. The simple fact is that the better football team does not always win, so plugging results of individual games into a formula is never going to be better than an eyeball test, or "beauty contest".

The eye test is inherently biased, but mathematical formulas don't have to be.

The bias in a computer formula is in the choices the programmer makes in terms of what factors to include in the formula, how to weight them, etc. And those choices reflect his subjective beliefs about what makes a team good. The biases are not skewed for or against particular schools or conferences, but pertain to abstract team profiles.

That's why it's best to use composites of multiple computers, so that different biases in different formulas might mitigate each other.

But it still doesn't compensate for the small sample size and limited comparative schedules (most games are within a conference). For the Big 12, they only have 30 games out of conference and usually 8-9 of those are against FCS teams.

I've seen some researchers work wonders with sample sizes of 30. True, we don't get the same richness of data that we do with say hoops, but we do get enough to make good determinations. I mean, look at the computers, and then look at who we all know was good last year. There's a pretty strong correlation, unless you really do think that we don't know that Ohio State was better than Memphis and Memphis was better than USF.

I have done the analysis that I referred to in my Post #15 several times in recent years. The results have always been strikingly similar. At the top - the top dozen or so teams - the computer models are pretty good. But they are also something else - they are nearly identical to the human polls, which include the AP (writers), Coaches, and CFP selection committee poll. That's because those polls, by their nature, are a composite of the views of many humans who each bring their own "biases" (that is. their opinion about what makes one team better than another). That means they are no different than a composite of the biases of the mathematicians who reduced their biases into a formula that could be tested by a computer.

So, we seem to be striving to find something we already have, and have had for many years. The human polls are as good as it gets. Where those polls may be deficient is lower down in the rankings. I suspect that is because those voters stop being concerned about which teams are the "best", that is to say the rankings that have actual consequences like invitations to NY6 bowls, and start throwing bones to teams they feel deserve to have their good seasons recognized, even if they aren't really the 20th best team in all the land. There, I believe the computer models may be more useful.
(This post was last modified: 06-27-2020 09:00 AM by ken d.)
06-27-2020 08:54 AM
Find all posts by this user Quote this message in a reply
Post Reply 




User(s) browsing this thread: 1 Guest(s)


Copyright © 2002-2024 Collegiate Sports Nation Bulletin Board System (CSNbbs), All Rights Reserved.
CSNbbs is an independent fan site and is in no way affiliated to the NCAA or any of the schools and conferences it represents.
This site monetizes links. FTC Disclosure.
We allow third-party companies to serve ads and/or collect certain anonymous information when you visit our web site. These companies may use non-personally identifiable information (e.g., click stream information, browser type, time and date, subject of advertisements clicked or scrolled over) during your visits to this and other Web sites in order to provide advertisements about goods and services likely to be of greater interest to you. These companies typically use a cookie or third party web beacon to collect this information. To learn more about this behavioral advertising practice or to opt-out of this type of advertising, you can visit http://www.networkadvertising.org.
Powered By MyBB, © 2002-2024 MyBB Group.