(06-16-2021 11:42 AM)Fighting Muskie Wrote:  Crayton— having you ever gone back and compared your sims to what actually happens? I’m curious what percentage of games have the same outcomes as your sim.

The only comparisons I've done have involved the Playoff/NY6, comparing those predictions across the regular season with other publishers'. Not many calculate the likelihood of teams making the playoff. I blew fivethirtyeight out of the water and performed just below Massey. I incorporated point data only 2 years ago (used just W/L before that) in the hopes of making the model better, but Massey stopped publishing with Washington Post that year.

I was ready to compare against a betting service last year and ESPN started including the chances in their FPI last year, but that data disappeared mid-season and few computers did well because of COVID. I have not yet explored preseason predictions to compare against this year; hopefully FPI data comes back; I'll likely spend less time on this in the Fall as I'm going back to school.

I could probably get a correct W/L % pretty easy from my data (technically I have data back to '08, but those files may now be incompatible). With so many lopsided games, I expect to score ~70%; hopefully higher. Congrove compares their predictions against the spread (they average 75% straight-up W/L and 53% against the spread), but it'd take a lot of work to skim Vegas data.