Forecast evaluation tests in the presence of ARCH

Abstract

The practice of forecasting is fundamental to economic analysis, and substantial importance is often attached to the predictions obtained. As a consequence, detailed evaluation of economic forecasts is crucial. Forecast evaluation is frequently conducted by comparing competing forecasts, and it is important therefore to have available formal testing procedures enabling such comparisons to reliably be made. Within this framework, a common approach involves testing the hypothesis of equal forecast mean squared error (or some other accuracy measure). In an important contribution to the literature on forecast evaluation, Diebold and Mariano (1995) proposed a general method for testing the equality of competing forecasts' accuracy. Given two competing h-steps-ahead forecasts, denote by (e 1t , e 2t ) the corresponding errors. Quite generally, we can consider tests of the null hypothesisDiebold and Mariano (1995) concentrate on cases where the function f(.) in equation (1) , the null is that the two forecasts have equal mean squared error.A second approach in comparing rival predictors is to test for forecast encompassing. According to the de®nition of Chong and Hendry (1986), a set of forecasts is said to encompass a competing set of forecasts if the latter should optimally receive zero weight in a composite predictor that is a weighted average of the two individual predictors. This more rigorous

Novel machine learning insights into the QM7b and QM9 quantum mechanics datasets

“…The model quality measures are better than the ones obtained when using all features (Table 7), indicating that there are irrelevancies, noise, and harmful interactions within the set of original properties. This is confirmed by the results of a Diebold‐Mariano test, 64,65 between the prediction of the number of atoms of each chemical element given by the model using all features, and the one using the 10 selected features (Table 11).…”

STAR and ANN models: forecasting performance on the Spanish “Ibex-35” stock index

“…We also reject the null hypothesis for all models against the Elman model at 5%. Finally, for Panel C, we reject the null hypothesis in Elman net against other models (except the random walk model).8 It should be noted that when a finite sample (i.e., H=100) is utilised,Harvey et al (1998Harvey et al ( , 1999 recommend modifying the DM test. They prefer to compare the modified Diebold and Mariano statistic with critical values from the t-Student distribution with HÀ1 degrees of freedom, rather than use the normal standard.…”

On the short-term predictability of exchange rates: A BVAR time-varying parameters approach

“…Harvey et al (1997a) argue that the Diebold-Mariano test tends to be seriously oversized, particularly for moderate samples, when the forecast errors are generated by heavytailed distributions. In a further study, Harvey et al (1997b) show that the presence of autoregressive conditional heteroscedasticity (ARCH) effects in forecast errors can induce size distortions in the Diebold-Mariano test. To overcome these problems, the authors propose the following modified Diebold-Mariano test, LS * ,…”