Gauging the uncertainty of the economic outlook using historical forecasting errors: The Federal Reserve’s approach

Abstract

Since November 2007, the Federal Open Market Committee (FOMC) of the U.S. Federal Reserve has regularly published participants' qualitative assessments of the uncertainty attending their individual forecasts of real activity and inflation, expressed relative to that seen on average in the past. The benchmarks used for these historical comparisons are the average root mean squared forecast errors (RMSEs) made by various private and government forecasters over the past twenty years. This paper documents how these benchmarks are constructed and discusses some of their properties. We draw several conclusions. First, if past performance is a reasonable guide to future accuracy, considerable uncertainty surrounds all macroeconomic projections, including those of FOMC participants. Second, different forecasters have similar accuracy. Third, estimates of uncertainty about future real activity and interest rates are now considerably greater than prior to the financial crisis; in contrast, estimates of inflation accuracy have changed little. Finally, fan charts-constructed as plus-or-minus one RMSE intervals about the median FOMC forecast, under the expectation that future projection errors will be unbiased and symmetrically distributed, and that the intervals cover about 70 percent of possible outcomes-provide a reasonable approximation to future uncertainty, especially when viewed in conjunction with the FOMC's qualitative assessments. That said, an assumption of symmetry about the interest rate outlook is problematic if the expected path of the federal funds rate is expected to remain low.

Real-Time Density Nowcasts of US Inflation: A Model-Combination Approach

“…Estimates of survey density nowcasts based on historical errors have been shown to be a good benchmark, especially for inflation. 37 The Federal Open Market Committee uses historical forecast errors to provide an estimate of the uncertainty surrounding the outlook in the Summary of Economic Projections (see Reifschneider and Tulip, 2019). While the SPF does provide some density forecasts by combining individual respondents' density forecasts, we favor the historical errors approach because Clements (2018) shows that the survey projection's second moments are inferior to simple statistical models.…”

Censored Density Forecasts: Production and Evaluation

“…Many forecaster and central bank assessments of future uncertainties are informed, at least in part, by monitoring past forecast errors. 1 As Reifschneider and Tulip (2019) review, this is the general approach to gauging unconditional forecast uncertainty at the US Federal Reserve, the European Central Bank, the Bank of England, the Reserve Bank of Australia, the Bank of Canada, and the Swedish Riksbank.…”

“…While the Federal Reserve and the CBO assume normality, they do not emphasize the distributional form of the inner confidence interval, as they publish only a 70 percent interval and do not show intervals within this. In estimating this normal density, as explained by Reifschneider and Tulip (2019), the FOMC uses errors from the last 20 years of forecasts. This has the implication that the large errors associated with the recession of 2008-09 and the pandemic of 2020 should be expected about once in 10 years.…”

Modeling Time-Varying Uncertainty of Multiple-Horizon Forecast Errors

“…For example, in their analysis of historical forecast accuracy (work that underlay the Federal Reserve's initial publication of forecast accuracy measures in the SEP), Reifschneider and Tulip (2007) explicitly chose a sample starting in 1986 to capture accuracy in the period since the start of the Great Moderation. The more recent analysis of Reifschneider and Tulip (2017) discusses some simple evidence of changes in the sizes of forecast errors. In practice, the historical accuracy measures published in the Federal Reserve's SEP are based on a 20-year window of forecast errors.…”

“… Tulip (2007, 2017) find a range of forecast sources, including SPF, Greenbook, and Blue Chip, to have similar accuracy of point forecasts.9 Studies such asFaust and Wright (2008) andReifschneider and Tulip (2017) make use of short-term interest rate forecasts from Greenbook obtained from the Federal Reserve's Board of Governors. However, as discussed in Wright (2008, 2009), for much of the available history, these forecasts have been tied to conditioning assumptions about monetary policy, rather than unconditional forecasts.…”