Who’s #1? The Science of Rating and Ranking by Amy N. Langville and Carl D. Meyer has been on my list for a while due to my work at SBLive. The book presents the mathematics behind today’s most widely used rating and rankings methods, which will of course come in handy when working on the sports rankings that I generate at work. I haven’t done any math much more complicated than balancing a checkbook since college, so I’m going to approach this book as a challenge.
Notes
Chapter One: Introduction to Ranking
- “The problem of ranking is elegantly simple — arrange a group of items in order of importance — yet some of its solutions can be complicated and full of paradoxes and conundrums.”
- “Early democracies, such as Greece, used the standard plurality voting system, in which each voter submits just one lone vote for their top choice.”
- “Plurality voting is contrasted with preference list voting in which each voter submits a preference list that places candidates in a ranked order. In this case, each voter creates a rank-ordered list of candidates. These ranked lists are somehow aggregated … to determine the overall winner.”
- “(Economist Kennth Arrow) found that his question about the existence of a perfect voting system has a negative answer”
- every voter can rank the candidates in any order, unrestricted domain criterion
- if a subset of voters always rank candidate A ahead of candidate B, then that rank order should be maintained when expanding back to the set of all candidates, independence of irrelevant alternatives
- if all voters choose A over B, then A should always rank ahead of B, pareto efficiency
- the book refers to this as the pareto principle which is something else, and wikipedia for this theorem uses the word “efficiency”
- no single voter should have the power to dictate the rankings, non-dictatorship
- “While Arrow’s four criteria seem obvious or self-evident, his result certainly is not. He proves that it is impossible for any voting system to satisfy all four common sense criteria simultaneously. Of course, this includes all existing voting systems as well as any clever new systems that have yet to be proposed.”
- “our running example uses data from the 2005 NCAA football season”
- “A ranking of items is a rank-ordered list of the items. Thus, a ranking vector is a permutation of the integers 1 through n.”
- “A rating of items assigns a numerical score to each item. A rating list, when sorted, creates a ranking list.”
Chapter Two: Massey’s Method
- “The Bowl Championship Series (BCS) is a rating system for NCAA college football that was designed to determine which teams are invited to play in which bowl games.”
- “In 1997, Kenneth Massey, then an undergraduate at Bluefield College, created a method for ranking college football teams. He wrote about this method, which uses the mathematical theory of least squares, as his honors thesis.”
- “In other words, the difference in the ratings $r_i$ and $r_j$ of the two teams ideally predicts the margin of victory in a contest between these two teams.”
- “The goal of any rating system is to associate a rating for each team in a league of n teams, where m total league games have been played thus far.”
- “$M$ need not be computed. It can simply be formed using the fact that diagonal element $M_i{}_i$ is the total number of games played by team $i$ and the off-diagonal element $M_i{}_j$, for $i \neq j$, is the negation of the number of games played by team $i$ against team $j$. … the right-hand side of the normal equations … can be formed by accumulating point differentials. The $i^t{}^h$ element of the right-hand side vector … is the sum of the point differentials from every game played by team $i$ that season.”
- “The Massey method uses point score data to rate teams. However, other game statistics could be used. And in other contexts, any data on pair-wise comparisons of items could be used. For example, to rate the trading power of nations, the gross number of exports could be used. The nation with the larger number of exports (perhaps normalized by accounting for population) would win in the hypothetical matchup.”
Rank: The rank is how many of the rows are “unique”, meaning not made of other rows. The same goes for columns. MathIsFun has a nice overview. Shape: The shape of a matrix is the number of rows and columns.
Chapter Resources
- Calculating Massey and Colley Ratings on NCAA Basketball Data
- This video walks you through the Massey Method, step by step.
- This post is someone doing the Massey method using Python
- Lecture notes from a Notre Dame class
Chapter Two: Colley’s Method
- “the Colley Rating Method, is a modification of one of the simplest and oldest rating systems, the rating system that uses winning percentage. Winning percentage rates team $i$ with the value $r_i$ according to the rule $r_i = w_i / t_i$ where $w_i$ and $t_i$ are the number of wins and total number of games played by team $i$, respectively.”
- “this rating system does have a few obvious flaws. First, ties in the ratings often occur in sports such as football where most teams play the same number of games against the same set of teams. Second, the strength of opponents is not factored into the analysis in any way. … Third, there are times when winning percentage ratings give unusual results. For instance, at the beginning of the season all teams have preseason ratings of $0/0$ and, in addition, as the season progresses a winless team has a rating of zero.”
- “The Colley method begins with a slight modification to the traditional winning percentage formula so that $r_i = 1 + w_i / 2 + t_i$ ”
[!question] Team Ratings Seeds
Colley’s Method assigns teams a starting value of 0.5. Would using our previous year’s data to assign a more accurate starting value help improve the accuracy of our results?