Uncertainty measures from partially rounded probabilistic forecast surveys

Abstract

Although survey‐based point predictions have been found to outperform successful forecasting models, corresponding variance forecasts are frequently diagnosed as heavily distorted. Professional forecasters who report inconspicuously low ex ante variances often produce squared forecast errors that are much larger on average. In this paper, we document the novel stylized fact that this variance misalignment is related to the rounding behavior of survey participants. Rounding may reflect the fact that some survey participants employ a rather judgmental approach to forecasting as opposed to using a formal model. We use the distinct numerical accuracies of panelists' reported probabilities as a way to propose several alternative and easily implementable corrections that (i) can be carried out in real time, that is, before outcomes are observed, and (ii) deliver a significantly improved match between ex ante and ex post forecast uncertainty. According to our estimates, uncertainty about inflation, output growth and unemployment in the U.S. and the Euro area is higher after correcting for the rounding effect. The increase in the share of nonrounded responses in recent years also helps to understand the trajectory of survey‐based average uncertainty during the years since the financial and sovereign debt crisis.

Heterogeneous Expectations among Professional Forecasters

“…Following this line of argument, Valchev and Gemmi (2023) suggest that survey forecasts may not reflect the true expectations of forecasters but might be driven by strategic incentives. While the literature almost exclusively focuses on disagreement in point predictions, disagreement in forecast uncertainty is an interesting area for future research (see Glas and Hartmann, 2022).…”

Expectation formation and the Phillips curve revisited

“…Therefore, to foster our understanding on the expectation formation mechanism, we extract individual information from distributional forecasts by following Abel et al (2016) and Glas and Hartmann (2022). In doing so, we rely on the "mass-at-midpoint" approach and compute the following measures:…”

“…In this context, it is also worth noting that the interior intervals in the ECB SPF have gaps of 0.1 percentage points between each other. These have been closed by extending the lower and upper bound of each interval by 0.05 following a convention in the existing literature (Abel et al 2016;Glas and Hartmann, 2022). To compute the midpoints m k , the intervals in both tails of the distribution have been assumed to have a width that is double as wide as the width of the interior intervals.…”

Constructing fan charts from the ragged edge of SPF forecasts

“…Using machine learning algorithms, Bianchi, Ludvigson, and Ma (2022) find evidence of time-varying bias in survey expectations and forecasts and conclude that artificial intelligence can be used to improve forecast accuracy. Regarding the predictive value of density forecasts collected by surveys, Clements (2018), as well as Glas and Hartmann (2022) and others, points to potential shortcomings, for example, due to rounding of answers by respondents.…”