Forecasting with a real-time data set for macroeconomists

Abstract

This paper discusses how forecasts are affected by the use of real-time data rather than latest-available data. The key issue is this: In the literature on developing forecasting models, new models are put together based on the results they yield using the data set available to the model's developer. But those are not the data that were available to a forecaster in real time. How much difference does the vintage of the data make for such forecasts? We explore this issue with a variety of exercises designed to answer this question. In particular, we find that the use of real-time data matters for some forecasting issues but not for others. It matters for choosing lag length in a univariate context. Preliminary evidence suggests that the span-or number-of forecast observations used to evaluate models may also be critical: we find that standard measures of forecast accuracy can be vintage-sensitive when constructed on the short spans (five years of quarterly data) of data sometimes used by researchers for forecast evaluation. The differences between using real-time and latest-available data may depend on what is being used as the "actual" or realization, and we explore several alternatives that can be used. Perhaps of most importance, we show that measures of forecast error, such as root-mean-squared error and mean absolute error, can be deceptively lower when using latest-available data rather than real-time data. Thus, for purposes such as modeling expectations or evaluating forecast errors of survey data, the use of latest-available data is questionable; comparisons between the forecasts generated from new models and benchmark forecasts, generated in real time, should be based on real-time data.

Nowcasting GDP in Real Time: A Density Combination Approach

“…A key issue in this exercise is the choice of a benchmark for the "actual" measure of GDP. Stark and Croushore (2002) discuss three alternative benchmark data vintages: the most recent data vintage, the last vintage before a structural revision (called a benchmark vintage) and finally the estimate that is released a fixed period of time after the first release. We follow Romer and Romer (2000) in using the second available estimate of GDP as the actual measure.…”

A note on the representative adaptive learning algorithm

“…Our data on these series comes from the Philadelphia's Fed Real-Time Data Research Center and consists of vintages from 1966q1 to 2012q1, i.e., a total of 185 snapshots of what was known on these variables by a market participant in real-time (see Stark and Croushore, 2002). For the purpose of comparing the algorithms forecasts to those provided by survey respondents, we use data from the Survey of Professional Forecasters (SPF).…”

Real-time forecasting US GDP from small-scale factor models

“…Among many others, Stark and Croushore (2002) suggest that the analysis of the in-sample forecasting performance of competitive models is questionable since the results can be deceptively lower when using real-time vintages. This happens because the in-sample analysis misses three aspects of real-time forecasting: (i) the recursive estimation of the model parameters; (ii) the real time data flow, i.e.…”