Ranking in Incomplete Tournaments: The Generalized Win Percentage Method, Efficiency, and NCAA Football

Abstract

I present a parsimonious family of scoring methods for ranking teams in incomplete tournaments. It is especially useful if teams play different number of games. I show that this family of scoring methods satisfies common sense theoretical properties studied by the literature on paired comparisons. I analyze it in terms of efficiency, defined as how close a scoring method comes to capturing what the teams’ win percentages would have been, in a complete tournament. Finally, using data on betting odds, I calibrate the family of scoring methods to match, as closely as possible, the actual rankings that were used to determine the teams that would go on to compete for the championship of the NCAA division 1 football tournament between 2011 and 2023. I find that the rankings used by the NCAA were generally efficient and I quantify the biases present in each year’s ranking.