The only information used in the ratings and predictions is that provided by knowing the scores of past games (and which of those games were decided in overtime) and by knowing the date and location of each game and the identity of the opposing teams. In fact, the dependence on the score of each past game is only through the difference in score between whichever team is the “home team” and whichever team is the “visiting team” or, equivalently, through the identity of the winning team and the MOV (margin of victory).
The methodology used in generating the ratings and predictions was devised from a model-based statistical approach. For a description of the model and a discussion of its use in generating ratings and predictions, refer to the following publication and to various of the references therein: “The Need for More Emphasis on Prediction: A “Nondenominational” Model-Based Approach,” The American Statistician, Volume 68, Pages 71–92.
In the past, the model has typically been applied directly to the differences in score. Such an approach does not differentiate between the outcomes of games decided in OT (overtime) and those decided in regulation play, and it does not account for the “lumpiness” in the distribution of scores documented by Mosteller in the following article: “Collegiate Football Scores, U.S.A.” Journal of the American Statistical Association, Volume 65, Pages 35–48, Allowances were made (in the methodology used to generate the ratings and predictions) for the existence of OT play and for the lumpiness of the distribution of scores; this was accomplished by adopting a modified version of the model-based approach in which the model is applied to unobservable “latent variables” rather than to the differences in score themselves.
The methodology allows for some flexibility. In what follows, various aspects of the methodology are described (both in general terms and in the case of the specific implementation employed during the 2018 season).
Data. In implementing the methodology, a decision obviously has to be made as to whether the ratings and/or predictions are to be based on the outcomes of “all” of the past games or on only the outcomes of a designated subset of games. In the 2018 implementation, the emphasis is on rating I-FBS teams and on predicting the outcomes of games involving I-FBS teams. Accordingly, the past games whose outcomes are used are confined to those among Division-I teams (both I-FBS and I-FCS). Further, there is a restriction to whatever games have already been played during the 2018 season and to games played during the 3 preceding seasons.
Groups. The teams can be partitioned into groups (with the objective of forming a relatively small number of “relatively uniform” groups). When it comes to the various characteristics of the model, teams within the same group are regarded as “interchangeable.” In particular, the “effects” of any particular team are modeled as random variables whose distribution is the same as that of other teams in the same group—the relationship among the effects of any particular team in different years is taken to be that associated with a first-order autoregressive process. One of the groups may be regarded as the “base group” and the ratings expressed so as to be interpretable as deviations from the “level” of an “average” team in the base group. In the 2018 implementation, the teams were partitioned into 3 groups as follows: (1) power-5 I-FBS teams (which was regarded as the base group); (2) non-power-5 I-FBS teams; and (3) I-FCS teams.
Cap on MOV and Ephasis on Winning Per Se. The methodology is such that the effect of MOV can be capped at a specified number of points (in the sense that if the outcome of a game is such that the cap is exceeded, the amount by which it is exceeded has no effect on the ratings or predictions). And the methodology is such that a weight (between 0 and 1, inclusive) can be assigned to “winning per se.” In the 2018 implementation, a weight of 0.35 was assigned to winning per se. And initially (in generating the 07/28/18 and 08/28/18 ratings and predictions) a cap of 45 points was placed on MOV. However, it subsequently became clear that early in the season the imposition of a cap on MOV can result in a significant loss of information in regard to yearly changes in the performance levels of various teams. Consequently, beginning with the 09/04/18 ratings and predictions, the cap on MOV was dropped.
Underlying Distribution. The distribution of the model’s residual effects can be taken to be a standard normal distribution. Alternatively, it can be taken to be some other symmetric distribution, perhaps one with longer tails. In the 2018 implementation, this distribution was initially taken to be a Student’s t distribution with 3 degrees of freedom—the motivation for this choice came from results obtained during the preceding (2017) season. Beginning with the 09/23/18 ratings and predictions (and based on an evaluation of the characteristics of the predictions made in the preceding weeks of the 2018 season), a switch was made to a Student’s t distribution with 5 degrees of freedom. Subsequently, beginning with the 10/14/18 ratings and predictions (and based on similar considerations) a switch was made to a Student’s t distribution with 7 degrees of freedom. And the following week (i.e., beginning with the 10/21/18 ratings and predictions) a switch was made to a standard normal distribution. Then, starting with the first week of the 2019 season, there was a return to the use of a longer-tailed distribution; in this case, the longer-tailed distribution was one determined empirically from the information that had been acquired about the residual effects associated with previous games.
A Correction. Subsequent to the production of the 11/25/18 ratings and predictions, an error was discovered in one of the underlying procedures. The effect of this error was to place too much weight on the outcomes of games from previous seasons. Prior to the production of the 12/02/18 ratings and predictions, a correction was made (and some unrelated “refinements” were introduced).