Testing Bayesian Updating With the Associated Press Top 25

Abstract

Most studies of Bayesian updating use experimental data. This article uses a non-experimental data source-the voter ballots of the Associated Press college football poll, a weekly subjective ranking of the top 25 teams-to test Bayes' rule as a descriptive model. I find that voters sometimes underreact to new information, sometimes overreact, and at other times their behavior is consistent with estimated Bayesian updating. A unifying explanation for the disparate results is that voters are more responsive to information that is more salient (i.e., noticeable). In particular, voters respond in a "more Bayesian" way to losses and wins over ranked teams, as compared to wins over unranked teams, and voters seem unaware of subtle variation in the precision of priors. (JEL D80, D83, D84) 2. There are a number of other fairly well-known criticisms of experimental work. Levitt and List (2007) provide an interesting discussion of some of these issues, including self-selection of subjects, small stakes, self-consciousness, inability to confer with others, and insufficient time to make optimal decisions. 3. Several other academic studies have used the AP Top 25 as a data source, including Goff (1996), Lebovic and Sigelman (2001), and Logan (2010). None focus on analyzing the rationality of belief updating.

Discrimination and Information: Geographic Bias in College Basketball Polls

“…This may occur because pollsters are concerned that, even if they know a more-nearby team to be of high quality, publicly voting them to a high rank may reek of taste-based bias [Morris 2001]. Also, if local pollsters possess more information about more-nearby teams, they may overreact to such information [Stone 2013] and, for example, punish a local team too severely if it loses or plays poorly in a recent game.…”

“…That the preference for proximity is more intense at the bottom of ballots may be because pollsters' discernment of team quality is significantly worse at lower ranks than at higher ranks [Nutting 2011;Stone 2013], and pollsters may allow their geographic biases to play larger roles when ordinal rankings are less clear. Another factor is that negative repercussions of bias may be lessened at lower ranks.…”

“…The RANK controls in Equation (4) are both quadratic in form since poll accuracy is worse at lower ranks [Nutting 2011;Stone 2013]. When the worse-ranked team is unranked, RANK wgpt is represented by a dummy variable.…”

Do we follow others when we should outside the lab? Evidence from the AP top 25

“…The primary one is that the voters' objective function(s) is/are unclear. The guidelines the AP gives voters on how to rank teams are intentionally vague and meant to allow for subjectivity (Stone 2013). We deal with this issue by, first, assuming the voters have a particular objective function-to rank teams such that, ceteris paribus, higher ranked teams are likely to perform better than lower ranked teams in future games.…”

“…This approach can be thought of as testing the internal consistency of voter behavior, a necessary, but not sufficient, condition for optimal information processing. We use two methods to analyze whether voters meet this condition; the second is an adaptation of the method of Stone (2013), in which the voters' optimal posterior ranks are directly estimated.…”