Uncertainty Measures from Partially Rounded Probabilistic Forecast Surveys

Abstract

Although survey-based point predictions have been found to outperform successful forecasting models, corresponding variance forecasts are frequently diagnosed as heavily distorted. Forecasters who report inconspicuously low ex-ante variances often produce squared forecast errors that are much larger on average. In this paper, we document the novel stylized fact that this variance misalignment is related to the rounding behavior of survey participants. Rounding may reflect the fact that some survey participants employ a rather judgmental approach to forecasting as opposed to using a formal model. We use the distinct numerical accuracies of panelists' reported probabilities as a means to propose several alternative and easily implementable corrections that i) can be carried out in real time, i.e., before outcomes are observed, and ii) deliver a significantly improved match between ex-ante and ex-post forecast uncertainty. According to our estimates, uncertainty about inflation, output growth and unemployment in the U.S. and the Euro area is higher after correcting for the rounding effect. The increase in the share of non-rounded responses in recent years also helps to understand the trajectory of survey-based average uncertainty during the years since the financial and sovereign debt crisis.

Testing for Differences in Survey-Based Density Expectations: A Compositional Data Approach

“…A number of studies have recently observed that one can distinguish two types of surveybased forecasts based on the rounding behavior of the panelists (Binder, 2017;Clements, 2021;Glas and Hartmann, 2022;Reiche and Meyler, 2022). For density forecasts, Glas and Hartmann (2022) show that one type of panelists ('rounders') state interval probabilities that are multiples of five or ten and tend to assign positive probabilities to only a relatively small subset of the surveyed bins while another type of panelists ('non-rounders') reports probabilities that do not share a common divisor and tend to consider a larger number of bins, often reporting probabilities with higher precision (i.e., to at least one decimal point) for most, or indeed, all of the surveyed bins.…”

“…We define rounders as those panelists who report histograms containing probabilities of which more than half are multiples of five. Glas and Hartmann (2022) document that these two groups of forecasters differ in terms of the variances of their probabilistic forecasts. As shown in Section 3, the Hotelling test is able to detect such differences in second moments even if they are small.…”

“…In recent years, the use of survey-based expectation data has become increasingly more common. This process is particularly strong in macroeconomics where more and more surveys are set up to study the macroeconomic expectations of private households (D'Acunto et al, 2021;Conrad et al, 2022), firms (Coibion et al, 2018(Coibion et al, , 2020Andrade et al, 2022), and professional forecasters (Rich and Tracy, 2021;Glas and Hartmann, 2022).…”

A Bayesian Approach to Inference on Probabilistic Surveys

“…7 Third, bounded distributions such as the beta or the mixture of uniforms take literally the z ij that are zero, in that they place no probability mass on bins where the respondents place no mass. More broadly, the approach outlined in expression (1) does not directly address the issue of rounding: it solves the minimization problem taking all the Z ij 's literally even though the respondent may be reporting approximate probabilities (Dominitz and Manski, 1996;D'Amico and Orphanides, 2008;Boero et al, 2008Boero et al, , 2014Engelberg et al, 2009;Manski and Molinari, 2010;Manski, 2011;Giustinelli et al, 2020;Glas and Hartmann, 2022, among others, discuss the issue of rounding; Binder, 2017 uses rounding to measure uncertainty). 8 There have been attempts…”

Uncertainty measures from partially rounded probabilistic forecast surveys

“…1414 Figure A.1 in the Online Supplementary Material (Glas and Hartmann (2022)) shows that average uncertainty based on the variances from the beta distributions, denoted as , tends to be smaller in magnitude than that based on the mass‐at‐midpoint assumption. However, the break in the inflation uncertainty series is still present. …”