Comparing Predictions and Outcomes: Theory and Application to Income Changes

Abstract

Household surveys often elicit respondents' intentions or predictions of future outcomes. The survey questions may ask respondents to choose among a selection of (ordered) response categories. If panel data or repeated cross-sections are available, predictions may be compared with realized outcomes. The categorical nature of the predictions data, however, complicates this comparison. Generalizing previous findings on binary intentions data, we derive bounds on features of the empirical distribution of realized outcomes under the "best-case" hypothesis that respondents form rational expectations and that reported expectations are best predictions of future outcomes. These bounds are shown to depend on the assumed model of how respondents form their "best prediction" when forced to choose among (ordered) categories. An application to data on income change expectations and realizations illustrates how alternative response models may be used to test the best-case hypothesis.

Forecasting with panel data

“…For forecasting with micropanels, see Chamberlain and Hirano (1999), who suggested optimal ways of combining an individual's personal earnings history with panel data on the earnings trajectories of other individuals to provide a conditional distribution for this individual's earnings. Other applications to household survey data eliciting respondents' intentions or predictions for future outcomes, using panel data, include Keane and Runkle (1990) and Das et al (1999), among others. This survey does not get into the large literature on 'forecast combination methods' (see Diebold and Lopez, 1996;Newbold and Harvey, 2002;Stock and Watson, 2004; among others).…”

Testing the rationality of expectations of qualitative outcomes

“…This paper provides a test of subjective quantile or subjective mode rationality for predictions of categorical outcomes, using an asymptotic distribution for the minimum value of a set of moment inequalities (Kudo, ; Rosen, ). This test improves significantly upon previous approaches (such as Das et al., ) by testing all the necessary rationality conditions simultaneously. A Monte Carlo study (summarized in detail in the Supporting Information Appendix) shows that for a subjective mode hypothesis the new test differs from the previous single‐condition approaches, especially in scenarios that can have more than one mode.…”

“…Therefore one should not expect that the moments of binary expectations (such as the intention of doing something or not in the future) should be equal to the moments of the realized outcomes. Das, Dominitz, and Soest () extend Manski's analysis to the general case of multiple ordered categorical expectations of a variable with more than two values, showing that there is a difference between testing rationality under a subjective mode (i.e., the prediction with highest probability) or a subjective median (or another fixed quantile) assumptions. However, their approach tests several different moment inequalities separately and fails to provide an adequate confidence level of whether the conditions jointly hold.…”

Testing Consumers' Asymmetric Perception of Changes in Household Financial Situation

“…As it seems reasonable to relate the perception of the financial situation change to the a priori expectation of it, we decided to apply the methodology of Das et al. (1999).…”