11-28-2011, 12:29 PM
By Justin Lahart
College football fans have long had their doubts about the impartiality of coaches polls. A new analysis isn’t likely to change their minds.
Associated Press
Research conducted by Yale University economist Matthew Kotchen and University of Calif.-Santa Barbara political scientist Matthew Potoski, which covers the USA Today coaches poll administered by the American Football Coaches Association from 2005 to 2010, shows that coaches rank their own teams, teams in their own conference, and teams that they’ve defeated more favorably than merited. The researchers argue those biases skew the results of the poll, which is one of the components in the system used to determine which teams get to play in major bowl games, and what two teams go to the national championship game.
“The idea that coaches have been gaming the system is not new,” says Mr. Kotchen. “What we figured out is that you could use econometric methods to study this in a systematic way.”
Every week, about 60 college coaches submit ballots in the poll, which is administered by the AFCA. Those rankings are one of three components — the other two are the Harris poll of ex-players, ex-coaches, administrators and news media, and the average of six computer rankings — used to determine teams’ Bowl Championship Series standings.
Through much of the poll’s 62-year history, coach’s individual ballots were kept secret, but that changed after 2004, when California missed out on a Rose Bowl bid after Texas coach Mack Brown urged coaches to vote for his Longhorns. Since 2005, the coaches’ ballots in the final regular-season poll have been disclosed. Ballots from earlier in the season still aren’t made public.
AFCA executive director Grant Teaff declined to comment, saying he hasn’t seen the study.
Messrs. Kotchen and Potoski took the coaches’ individual ballots from 2005 through 2010 and performed two statistical analyses. In the first, they compared coaches’ rankings and the B.C.S. computer rankings, on the view that large differences between the two reflect coaches’ biases. In the second, they compared individual coaches’ ballots to the average of other coaches’ ballots — here, again, large differences reflect bias. Both analyses yielded the same conclusion: Coaches show a marked tendency to vote their own self-interest.
On average, they found that coaches boost the rankings of their own teams by more than two full positions while boosting the rankings of teams in their conference by nearly a full position. They upped the ranking of teams they’d beaten by more than half a position, making their own team look better.
When it comes to teams that are on the bubble of making it into one of the B.C.S. bowl games, the coaches tend to act in ways that will maximize the financial payoff to their own school. The way the bowl system works, if your team is from one of the six B.C.S. conferences (the champions of which receive automatic invitations to one of four bowl games), you share in the bowl-game revenues generated by teams in your conference. If you aren’t in a B.C.S. conference, you share in the revenues of any non-B.C.S. conference team that makes it to a bowl game. The two researchers found that B.C.S. conference coaches favored teams in their own conference. Non-B.C.S. coaches favored non-B.C.S. teams.
Mr. Kotchen says that for him the results don’t indicate anything nefarious is going; rather, he thinks that the coaches have subconscious biases that they can’t help but act on. Stanford coach David Shaw is naturally going to think well of his team, Alabama coach Nick Saban is going to highly regard other teams in the Southeastern Conference — maybe even Auburn — and Michigan State coach Mark Dantonio probably thinks that Michigan, which his team beat this year, is a pretty strong team.
That said, Messrs. Kotchen and Potoski also found evidence that coaches are much more biased when their ballots aren’t disclosed than when they are. A final statistical analysis they performed examined how much the coaches poll rankings changed from the penultimate week to the final week of regular season play. Since 2005 there’s been a much more marked tendency for the coaches’ rankings to move toward the computer-generated rankings than there was through 2004, when the coaches’ final-week ballots weren’t disclosed.
Those results suggest that coaches’ ballots should be disclosed throughout the season. But coaches have resisted doing that, and last year lobbied to remove all disclosure requirements from the poll. For Mr. Kotchen, that idea is a nonstarter: “It’s clear that the ballots should be publicly available at the end of the season,” he says.
http://blogs.wsj.com/economics/2011/11/2...-suggests/
College football fans have long had their doubts about the impartiality of coaches polls. A new analysis isn’t likely to change their minds.
Associated Press
Research conducted by Yale University economist Matthew Kotchen and University of Calif.-Santa Barbara political scientist Matthew Potoski, which covers the USA Today coaches poll administered by the American Football Coaches Association from 2005 to 2010, shows that coaches rank their own teams, teams in their own conference, and teams that they’ve defeated more favorably than merited. The researchers argue those biases skew the results of the poll, which is one of the components in the system used to determine which teams get to play in major bowl games, and what two teams go to the national championship game.
“The idea that coaches have been gaming the system is not new,” says Mr. Kotchen. “What we figured out is that you could use econometric methods to study this in a systematic way.”
Every week, about 60 college coaches submit ballots in the poll, which is administered by the AFCA. Those rankings are one of three components — the other two are the Harris poll of ex-players, ex-coaches, administrators and news media, and the average of six computer rankings — used to determine teams’ Bowl Championship Series standings.
Through much of the poll’s 62-year history, coach’s individual ballots were kept secret, but that changed after 2004, when California missed out on a Rose Bowl bid after Texas coach Mack Brown urged coaches to vote for his Longhorns. Since 2005, the coaches’ ballots in the final regular-season poll have been disclosed. Ballots from earlier in the season still aren’t made public.
AFCA executive director Grant Teaff declined to comment, saying he hasn’t seen the study.
Messrs. Kotchen and Potoski took the coaches’ individual ballots from 2005 through 2010 and performed two statistical analyses. In the first, they compared coaches’ rankings and the B.C.S. computer rankings, on the view that large differences between the two reflect coaches’ biases. In the second, they compared individual coaches’ ballots to the average of other coaches’ ballots — here, again, large differences reflect bias. Both analyses yielded the same conclusion: Coaches show a marked tendency to vote their own self-interest.
On average, they found that coaches boost the rankings of their own teams by more than two full positions while boosting the rankings of teams in their conference by nearly a full position. They upped the ranking of teams they’d beaten by more than half a position, making their own team look better.
When it comes to teams that are on the bubble of making it into one of the B.C.S. bowl games, the coaches tend to act in ways that will maximize the financial payoff to their own school. The way the bowl system works, if your team is from one of the six B.C.S. conferences (the champions of which receive automatic invitations to one of four bowl games), you share in the bowl-game revenues generated by teams in your conference. If you aren’t in a B.C.S. conference, you share in the revenues of any non-B.C.S. conference team that makes it to a bowl game. The two researchers found that B.C.S. conference coaches favored teams in their own conference. Non-B.C.S. coaches favored non-B.C.S. teams.
Mr. Kotchen says that for him the results don’t indicate anything nefarious is going; rather, he thinks that the coaches have subconscious biases that they can’t help but act on. Stanford coach David Shaw is naturally going to think well of his team, Alabama coach Nick Saban is going to highly regard other teams in the Southeastern Conference — maybe even Auburn — and Michigan State coach Mark Dantonio probably thinks that Michigan, which his team beat this year, is a pretty strong team.
That said, Messrs. Kotchen and Potoski also found evidence that coaches are much more biased when their ballots aren’t disclosed than when they are. A final statistical analysis they performed examined how much the coaches poll rankings changed from the penultimate week to the final week of regular season play. Since 2005 there’s been a much more marked tendency for the coaches’ rankings to move toward the computer-generated rankings than there was through 2004, when the coaches’ final-week ballots weren’t disclosed.
Those results suggest that coaches’ ballots should be disclosed throughout the season. But coaches have resisted doing that, and last year lobbied to remove all disclosure requirements from the poll. For Mr. Kotchen, that idea is a nonstarter: “It’s clear that the ballots should be publicly available at the end of the season,” he says.
http://blogs.wsj.com/economics/2011/11/2...-suggests/