Evaluating a three-dimensional panel of point forecasts: The Bank of England Survey of External Forecasters

Abstract

This article provides a first analysis of the forecasts of inflation and GDP growth obtained from the Bank of England's Survey of External Forecasters, considering both the survey average forecasts published in the quarterly Inflation Report, and the individual survey responses, recently made available by the Bank. These comprise a conventional incomplete panel dataset, with an additional dimension arising from the collection of forecasts at several horizons; both point forecasts and density forecasts are collected. The inflation forecasts show good performance in tests of unbiasedness and efficiency, albeit over a relatively calm period for the UK economy, and there is considerable individual heterogeneity. For GDP growth, inaccurate real-time data and their subsequent revision are seen to cause serious difficulties for forecast construction and evaluation, although the forecasts are again unbiased. There is evidence that some forecasters have asymmetric loss functions.

Accuracy, unbiasedness and efficiency of professional macroeconomic forecasts: An empirical comparison for the G7

“…These include Keane and Runkle (1990) and Batchelor and Dua (1990), who introduce an analysis in a panel framework using the Generalized Methods of Moments (GMM) method, or Davies and Lahiri (1995), who develop a framework for analyzing three-dimensional panels of survey data, enabling the use of information along all dimensions. To ensure that our results are comparable to existing studies, we closely follow the approach suggested by Davies and Lahiri (1995), and recently used by Clements et al (2007), Boero et al (2008a), and Ager et al (2009), and suggest only minor modifications to the econometric framework.…”

“…A forecaster could, for example, predict a specific event to provoke a policy action that actually prevents the occurrence of the event. Third, forecasters might have asymmetric loss functions (Capistran and Timmermann, 2006, Boero et al, 2008a. These could have different weights concerning a possible over-or underestimation of an outcome.…”

Adjusting for information content when comparing forecast performance

“…We also assume a publication lag of one month, which means that the December outcome becomes available in January. 7 A somewhat similar decomposition for analyzing forecast errors in a panel data setting has been used by Davies and Lahiri (1995), Davies (2006) and Boero et al (2008) with the difference that they do not adjust for timing.…”

Analyzing Three-Dimensional Panel Data of Forecasts

“…This hypothesis has been overwhelmingly rejected in the studies of inflation forecasts, cf. Davies and Lahiri (1999) and Boero, et al (2008). Thus, the uncertainty measure constructed according to (0.37) will necessarily underestimate the "true" ex post uncertainty.…”