State‐dependent evaluation of predictive ability

Siliverstovs, Boriss; Wochner, Daniel

doi:10.1002/for.2715

Cited by 3 publications

(5 citation statements)

References 42 publications

(207 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Models with weekly data indeed show greater predictive accuracy compared with an AR model during “crisis” episodes but have similar performances during “normal” periods. This is in line with the literature pointing out asymmetries in forecasting performance across expansions and recessions, for example, Chauvet and Potter (2013) or Siliverstovs and Wochner (2021) who test a wide range of specifications and find that such asymmetries in forecasting performance across the business cycle phases are rather common. Our results notably confirm Siliverstovs (2021) showing that though the New York FED's nowcasting models are at least as good as an AR model during expansions, they entail substantial gains in accuracy during recessions.…”

Section: Introductionsupporting

confidence: 90%

“…A recent strand of literature has documented state-dependent performances of forecasting models (Chauvet & Potter, 2013;Siliverstovs, 2020Siliverstovs, , 2021Siliverstovs & Wochner, 2021), which can differ depending on the state of the business cycle such as recessionary versus expansionary period. The possibility of asymmetries in forecasting performances is even greater with high-frequency data given the general trade-off between timeliness and accuracy (Ahnert & Bier, 2001): Such data provide a timely signal that can enhance nowcasting performance if economic conditions suddenly deteriorate, but during "normal" periods, their contribution might be only of second order.…”

Section: "Crisis" Versus "Normal" Periodsmentioning

confidence: 99%

See 1 more Smart Citation

Nowcasting world GDP growth with high‐frequency data

Jardet

Meunier

2022

Journal of Forecasting

View full text Add to dashboard Cite

The Covid-19 crisis has shown how high-frequency data can help tracking economic turning points in real-time. Our paper investigates whether high-frequency data can also improve the nowcasting performances for world GDP growth on quarterly or annual basis. To this end, we select a large dataset of 151 monthly and 39 weekly series for 17 advanced and emerging countries representing 68% of world GDP. Our approach builds on a Factor-Augmented MIxed DAta Sampling (FA-MIDAS) which allows us to take advantage of our large database and to combine different frequencies. Models that include weekly data significantly outperforms other models relying on monthly or quarterly indicators, both in-and out-ofsample. Breaking down our sample, we show that models with weekly data have similar nowcasting performances relative to other models during "normal" times but strongly outperform them during "crisis" episodes (2008-2009 and 2020). We finally construct a nowcasting model of annual world GDP growth incorporating weekly data which give timely (one every week) and accurate forecasts (close to IMF and OECD projections, but with a 1 to 3 months lead). Policy-wise, this model can provide an alternative "benchmark" projection for world GDP growth during crisis episodes when sudden swings in the economy make the usual "benchmark" projections (from the IMF or the OECD) rapidly outdated.

show abstract

Section: Introductionsupporting

confidence: 90%

Section: "Crisis" Versus "Normal" Periodsmentioning

confidence: 99%

Nowcasting world GDP growth with high‐frequency data

Jardet

Meunier

2022

Journal of Forecasting

View full text Add to dashboard Cite

show abstract

“…It has been repeatedly reported that the benefits of a large panel of predictors may solely be present during periods of economic turmoil (Kotchoni et al, 2019; Siliverstovs and Wochner, 2019). For this reason and others (Lerch et al, 2017), it is of interest to study the marginal benefits associated with data-rich models outside of the tumultuous entry/exit of the Great Recession and the Pandemic Recession.…”

Section: Resultsmentioning

confidence: 99%

Can Machine Learning Catch the Covid-19 Recession?

Coulombe

Marcellino

Stevanović

2021

NIER

View full text Add to dashboard Cite

Based on evidence gathered from a newly built large macroeconomic dataset (MD) for the UK, labelled UK-MD and comparable to similar datasets for the United States and Canada, it seems the most promising avenue for forecasting during the pandemic is to allow for general forms of nonlinearity by using machine learning (ML) methods. But not all nonlinear ML methods are alike. For instance, some do not allow to extrapolate (like regular trees and forests) and some do (when complemented with linear dynamic components). This and other crucial aspects of ML-based forecasting in unprecedented times are studied in an extensive pseudo-out-of-sample exercise.

show abstract

“…The aim of this exercise was to replicate the study of Stock and Watson (2002) on a more recent data vintage but evaluate the forecasting performance of the diffusion-index model separately for the NBER expansions and recessions in a similar way as done in Chauvet and Potter (2013). Siliverstovs and Wochner (2021) confirm that there are systematic differences in forecasting accuracy across the business cycle phases both in absolute and relative terms with respect to the benchmark models. During expansions, both diffusion-index models and benchmark models generally display similar forecasting performance.…”

Section: Literature Reviewmentioning

confidence: 95%

“…The findings of Chauvet and Potter (2013) and Siliverstovs (2020a), reported for a single time series (US GDP growth), were extended in Siliverstovs and Wochner (2021) for each time series in the Stock-Watson dataset comprising more than 200 US macroeconomic variables. The aim of this exercise was to replicate the study of Stock and Watson (2002) on a more recent data vintage but evaluate the forecasting performance of the diffusion-index model separately for the NBER expansions and recessions in a similar way as done in Chauvet and Potter (2013).…”

Section: Literature Reviewmentioning

confidence: 99%

New York FED Staff Nowcasts and Reality: What Can We Learn about the Future, the Present, and the Past?

Siliverstovs

2021

Econometrics

Self Cite

View full text Add to dashboard Cite

We assess the forecasting performance of the nowcasting model developed at the New York FED. We show that the observation regarding a striking difference in the model’s predictive ability across business cycle phases made earlier in the literature also applies here. During expansions, the nowcasting model forecasts at best are at least as good as the historical mean model, whereas during the recessionary periods, there are very substantial gains corresponding in the reduction in MSFE of about 90% relative to the benchmark model. We show how the asymmetry in the relative forecasting performance can be verified by the use of such recursive measures of relative forecast accuracy as Cumulated Sum of Squared Forecast Error Difference (CSSFED) and Recursive Relative Mean Squared Forecast Error (based on Rearranged observations) (R2MSFE(+R)). Ignoring these asymmetries results in a biased judgement of the relative forecasting performance of the competing models over a sample as a whole, as well as during economic expansions, when the forecasting accuracy of a more sophisticated model relative to naive benchmark models tends to be overstated. Hence, care needs to be exercised when ranking several models by their forecasting performance without taking into consideration various states of the economy.

show abstract

State‐dependent evaluation of predictive ability

Cited by 3 publications

References 42 publications

Nowcasting world GDP growth with high‐frequency data

Nowcasting world GDP growth with high‐frequency data

Can Machine Learning Catch the Covid-19 Recession?

New York FED Staff Nowcasts and Reality: What Can We Learn about the Future, the Present, and the Past?

Contact Info

Product

Resources

About