Comparing forecast accuracy: A Monte Carlo investigation

Busetti, Fabio; Marcucci, Juri

doi:10.1016/j.ijforecast.2012.04.011

Cited by 205 publications

(21 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One could also easily adopt the more standard Diebold-Mariano (DM) test, which is less powerful and undersized. As Busetti and Marcucci (2013) showed, if one can reject the null of equal forecast accuracy with the simple DM test, there is no need to try more complicated tests based on bootstrapped or simulated critical values. The table shows the ratio of the root mean squared forecast error (RMSFE) of the model in each row to that of the VAR in levels, transformed to first differences.…”

Section: Forecasting Resultsmentioning

confidence: 99%

Forecasting multivariate time series under present-value model short- and long-run co-movement restrictions

Guillen

Hecq

Issler

et al. 2015

International Journal of Forecasting

View full text Add to dashboard Cite

a b s t r a c tUsing a sequence of VAR-based nested multivariate models, we discuss the different layers of restrictions that are imposed on the VAR in levels by present-value models (PVM hereafter) for series that are subject to present-value restrictions. Our focus is novel: we are interested in the short-run restrictions entailed by PVMs (Vahid & Engle, 1993, 1997 and their implications for forecasting.Using a well-known database, maintained by Robert Shiller, we implement a forecasting competition that imposes different layers of PVM restrictions. Our exhaustive investigation of several different multivariate models reveals that better forecasts can be achieved when restrictions are applied to the unrestricted VAR. Moreover, imposing short-run restrictions produces forecast winners 70% of the time for the target variables of PVMs and 63.33% of the time when all variables in the system are considered.

show abstract

Section: Forecasting Resultsmentioning

confidence: 99%

Forecasting multivariate time series under present-value model short- and long-run co-movement restrictions

Guillen

Hecq

Issler

et al. 2015

International Journal of Forecasting

View full text Add to dashboard Cite

show abstract

“…On the other hand, the DM test statistic is also negative for IWLS with SA initial values but not significant; and even though the DM test is not significant for the IWLS with random initial values, other test statistics suggest that IWLS forecasts with random initial values are significantly better than the RW forecasts. The significance level is lower with the ENC_F test which is, according to Busetti and Marcucci (2013), the most powerful of the given tests used here.…”

Section: Empirical Applicationmentioning

confidence: 91%

Estimation of STAR–GARCH Models with Iteratively Weighted Least Squares

Midiliç

2018

Comput Econ

View full text Add to dashboard Cite

This study applies the Iteratively Weighted Least Squares (IWLS) algorithm to a Smooth Transition Autoregressive (STAR) model with conditional variance. Monte Carlo simulations are performed to measure the performance of the algorithm, to compare its performance with the performances of established methods in the literature, and to see the effect of initial value selection method. Simulation results show that low bias and mean squared error are received for the slope parameter estimator from the IWLS algorithm when the real value of the slope parameter is low. In an empirical illustration, STAR-GARCH model is used to forecast daily US Dollar/Australian Dollar and FTSE Small Cap index returns. 1-day ahead out-of-sample forecast results show that forecast performance of the STAR-GARCH model improves with the IWLS algorithm and the model performs better that the benchmark model.

show abstract

“…Inoue and Kilian (2005) advocate the derivation of bootstrapped critical values, although Diebold (2013) argues that this is unnecessary and that the tabulated critical values will be sufficient. 19 A particular issue arises in the comparison of the forecasts from nested models since the forecasts will converge asymptotically and are likely to be cross-correlated even in finite samples, resulting in the test being severely undersized and lacking power if standard critical values are employed (see Clark and McCracken, 2001; Clark and West, 2006;Busetti and Marcucci, 2013).…”

Section: Statistical Comparisons Of Forecast Accuracymentioning

confidence: 99%

“…Building on earlier work byMcCracken (2007) and McCracken (2001, 2005a), Hansen and Timmerman develop an approach that modifies the p-values of tests of the null hypothesis of no predictability so that they become robust to sample-split-induced data-mining. A conceptually similar modification, albeit different in detail, is proposed byRossi and Inoue (2012) and further comparisons of the power of tests for the differences between out-of-sample forecasts can be found inBusetti and Marcucci (2013).…”

mentioning

confidence: 99%

Finite sample weighting of recursive forecast errors

Brooks

Burke

Stanescu

2016

International Journal of Forecasting

View full text Add to dashboard Cite

This paper proposes and tests a new framework for weighting recursive out-of-sample prediction errors in accordance with their corresponding in-sample estimation uncertainty. In essence, we show how as much information from the sample as possible can be used in the evaluation of prediction accuracy by commencing the forecasts at the earliest opportunity and weighting the prediction errors. We demonstrate through a Monte Carlo study that when only a small sample is available the proposed framework can select the correct model from a set of candidate models considerably more often than the existing standard approach. We also show that the proposed weighting approaches result in tests of equal predictive accuracy which have much better size than the standard approach. An application to a set of exchange rate data highlights relevant differences in the results of tests of predictive accuracy based on the standard approach versus the framework proposed in this paper.

show abstract

Comparing forecast accuracy: A Monte Carlo investigation

Cited by 205 publications

References 33 publications

Forecasting multivariate time series under present-value model short- and long-run co-movement restrictions

Forecasting multivariate time series under present-value model short- and long-run co-movement restrictions

Estimation of STAR–GARCH Models with Iteratively Weighted Least Squares

Finite sample weighting of recursive forecast errors

Contact Info

Product

Resources

About