Robust Forecast Evaluation of Expected Shortfall*

Abstract

Motivated by the Basel 3 regulations, recent studies have considered joint forecasts of Value-at-Risk and Expected Shortfall. A large family of scoring functions can be used to evaluate forecast performance in this context. However, little intuitive or empirical guidance is currently available, which renders the choice of scoring function awkward in practice. We therefore develop graphical checks (Murphy diagrams) of whether one forecast method dominates another under a relevant class of scoring functions, and propose an associated hypothesis test. We illustrate these tools with simulation examples and an empirical analysis of S&P 500 and DAX returns.

A joint quantile and expected shortfall regression framework

“…There are no positively homogeneous loss functions for the cases b ≥ 1. Our numerical simulations show that there is no gain in efficiency or numerical accuracy by deviating from the choice G 1 (z) = 0 (see also Nolde and Ziegel, 2017;Ziegel et al, 2017), which is also consistent with the homogeneity result. Consequently, we use G 1 (z) = 0 in the following.…”

“…Even though the ES is not elicitable stand-alone, show in their seminal paper that the quantile (the VaR) and the ES are jointly elicitable by introducing a class of joint loss functions, whose expectation is minimized by these two functionals. This joint elicitability result and the associated class of loss functions gives rise to a growing literature in both, joint estimation (Zwingmann and Holzmann, 2016) and in joint forecast evaluation (Acerbi and Szekely, 2014;Nolde and Ziegel, 2017;Ziegel et al, 2017) for the risk measures VaR and ES.…”

“…We evaluate the forecasting power of the VaR and ES of these models by the class of strictly consistent loss (scoring) functions for the VaR and ES of . We use Murphy diagrams introduced by Ehm et al (2016) and Ziegel et al (2017), which provide a parsimonious way to evaluate competing forecasts simultaneously for a full class of strictly consistent loss functions. In fact, one forecasting model significantly dominates another one with respect to the full class of strictly consistent loss functions if and only if the elementary score differences plotted in the Murphy diagrams are strictly negative (positive).…”

“…In fact, one forecasting model significantly dominates another one with respect to the full class of strictly consistent loss functions if and only if the elementary score differences plotted in the Murphy diagrams are strictly negative (positive). For further details on the theory and the implementation of Murphy diagrams, we refer to Ehm et al (2016) and Ziegel et al (2017). Figure 3 displays the average of the elementary score differences of the joint VaR and ES regression model against the three alternative models together with the respective 95% pointwise confidence bands for the elementary scores provided in Ziegel et al (2017) for the pair VaR and ES.…”

Forecast dominance testing via sign randomization

“…This suggests that intra-daily information encoded in realized volatility contains more predictive content than daily returns. 8 Similar conclusions were reached in [58]. As in the first case study, the results are qualitatively robust across the two test statistics T 1 , T 2 .…”