The only information used in my ratings of college football teams and in my predictions for the outcomes of college football games is that provided by knowing the scores of past games (and which of those games were decided in overtime), by knowing the date and location of each game, and by knowing the identities of the opposing teams. In fact, the dependence on the score of each past game is only through the difference in score between whichever team is the “home team” and whichever team is the “visiting team” or, equivalently, only through the identity of the winning team and the MOV (margin of victory).
The methodology used in generating the college football ratings and predictions was devised from a model-based statistical approach. For a fairly detailed description of the model and a discussion of its use in generating ratings and predictions, refer to the following publication and to various of the references therein: “The Need for More Emphasis on Prediction: A ‘Nondenominational’ Model-Based Approach,” The American Statistician, vol. 68, pp. 71–92. Something in the way of an overview is provided by the slides found at ISUslides; these are the slides from a presentation I gave in 2018 at The Conference on Predictive Inference and Its Applications (which took place in Ames, Iowa, on the campus of Iowa State University).
In the past, the model has typically been applied directly to the differences in score. Such an approach does not differentiate between the outcomes of games decided in OT (overtime) and those decided in regulation play, and it does not account for the “lumpiness” in the distribution of scores documented by Mosteller in the following article: “Collegiate Football Scores, U.S.A.” Journal of the American Statistical Association, vol. 65, pp. 35–48. Allowances were made (in the methodology used to generate the ratings and predictions) for the existence of OT play and for the lumpiness of the distribution of scores; this was accomplished by adopting a modified version of the model-based approach in which the model is applied to unobservable “latent variables” rather than to the differences in score themselves.
The methodology allows for some flexibility. In what follows, various aspects of the methodology are described (both in general terms and in the case of the implementation currently being employed).
Data. In implementing the methodology, a decision obviously has to be made as to which games are to be included among the past games on whose outcomes the ratings and predictions are to be based. In the past and current implementations, the emphasis has been on rating I-FBS teams and on predicting the outcomes of games involving I-FBS teams. Accordingly, the past games whose outcomes are used are confined to those among Division-I teams (both I-FBS and I-FCS). In recent implementations, games involving I-FCS teams from the Ivy League or the Pioneer League have not been included—these I-FCS teams are unlikely to play any I-FBS teams. Further, there is presently a restriction to games from earlier in the current season and to games from the 3 preceding seasons.
Groups. The teams can be partitioned into groups (with the objective of forming a relatively small number of “relatively uniform” groups). When it comes to the various characteristics of the model, teams within the same group are regarded as “interchangeable.” In particular, the “effects” of any particular team are modeled as random variables whose distribution is the same as that of other teams in the same group—the relationship among the effects of any particular team in different years is taken to be that associated with a first-order autoregressive process. One of the groups may be regarded as the “base group” and the ratings expressed so as to be interpretable as deviations from the “level” of an “average” team in the base group. In the current implementation and other recent implementations, the teams have been partitioned into 3 groups as follows: (1) power-5 I-FBS teams (which has been regarded as the base group); (2) non-power-5 I-FBS teams; and (3) I-FCS teams.
Emphasis on Winning Per Se. The methodology is such that a weight (between 0 and 1, inclusive) can be assigned to “winning per se.” In the current implementation and other recent implementations, a weight of 0.35 has been assigned to winning per se. In theory (i.e., if the model were perfect), placing some weight on winning per se results in a loss of information. However, in practice, the primary objective of each of the opposing teams is to win the game. And that is not directly reflected by anything in the model (though conceivably it could have some “effect” on the values of the thresholds underlying the latent variables), so that the actual loss of information could be less than the model would seem to indicate. Nevertheless, there are some (mostly non-conference) games in which one of the opposing teams (usually the visiting team) is much weaker than the other opposing team and is thought to be virtually certain to lose. Almost all of the information provided by the outcome of such a game lies in the difference in score. The current implementation of the rating and prediction methodology includes a “mechanism” for increasing the weight placed on MOV to an extent that depends on how “one-sided” the game is expected to be.
Underlying Distribution. In the current implementation and other recent implementations of the ratings and prediction methodology, the distribution of the model’s residual effects has been taken to be a standard normal distribution. Alternatively, this distribution could have been taken to be some other symmetric distribution, perhaps one with longer tails. In fact, in some past implementations, this distribution was taken to be a Student’s t distribution with a specified number (in the range 3 to 7,inclusive) of degrees of freedom. In a variation on previous practice (which was introduced and adopted some time ago in an attempt to make the methodology more “robust”), the distribution of the residual effects identified with future games was taken to be one that is determined empirically. More specifically, this distribution was taken to be one determined empirically from a large number of draws made from the posterior distribution of the residual effects identified with the past games.
Thresholds. The model makes allowances for MOVs as large as 100 points. The larger of these MOVs are unlikely to be represented in the database by more than a few games and the largest are unlikely to be represented by any games. Thus, the data provides little if any “direct” information about the values of the thresholds representing the upper and lower bounds of the interval of latent-variable values corresponding to any one of those MOVs. This situation is to be dealt with by assuming that each of the intervals includes the value of a random variable and that the joint distribution of these random variables is one identified with a situation where each of the intervals is of equal length—think of this assumption as representing “prior information.” Previously, the lack of information about some of the thresholds was dealt with in a more “ad hoc” manner.