Ranking Forecasts by Stochastic Error Distance, Information and Reliability Measures

Abstract

The stochastic error distance (SED) introduced by Diebold and Shin (2017) ranks forecast models by divergence between distributions of the errors of the actual and perfect forecast models. The basic SED is defined by the variation distance and provides a representation of the mean absolute error, but by basing ranking on the entire error distribution and divergence, the SED moves beyond the traditional forecast evaluations. First, we establish connections between ranking forecast models by the SED, error entropy and some partial orderings of distributions. Then, we introduce the notion of excess error for forecast errors of magnitudes larger than a tolerance threshold and give the SED representation of the mean excess error (MEE). As a function of the threshold, the MEE is a local risk measure. With the distribution of the absolute error as a prior for the threshold, its Bayes risk is the entropy functional of the survival function, which is a known measure in the information theory and reliability. Notions and results are illustrated using various distributions for the error. The empirical versions of SED, MEE and its Bayes risk are compared with the mean squared error in ranking regression and autoregressive integrated moving average models for forecasting bond risk premia.

Variants of Mixtures: Information Properties and Applications

“…We have developed a consistent estimate for M f (m(τ)) and applied it for comparing forecast models (Ardakani et al, 2018). We also have shown that M f (m(τ)) ≤ σ f , where σ f is the standard deviation and the equality holds if and onlyF is exponential.…”

“…This measure is used in Asadi et al (2017), Ardakani et al (2018), andArdakani et al (2020), where it is referred to as the Bayes risk of m(τ), which is a misnomer. The Bayes risk of m(τ) is given by the E τ [Var(X − τ|X > τ)].…”

A Lightweight Processing Method for Hand-Drawn Pressure-Sensitive Trajectories Oriented Toward Web-Based 3D Modeling

“…We believe that the stroke speed of 300 px/sec is closest to the normal speed of hand-drawing. The experimental metrics shown in Table 5 were used to measure the simplification effect, which includes the compression ratio (CR), local maximum error (E max ), total length error (E l ), mean stochastic error distance (SED) [53], [54], and whole processing time t. Note that the metric t includes the whole process from trajectory data acquisition to trajectory data upload, followed by the trajectory data download and visualization in the Web browser. We show the results of the data simplification experiments for four hand-drawn trajectories using the 6 different methods shown in Table 6.…”

“…To further simplify the trajectory data, the resampled discrete points are simplified by comprehensively considering the angle deviation and chord height deviation with the sliding window mechanism, which can make the trajectory data lightweight and achieves a high degree of accuracy. To quantitatively evaluate the feasibility and effect of trajectory compression, some metrics are used to measure the data reduction effect which include compression ratio (CR) and whole processing time t, while others are used to measure the accuracy effect which include local maximum error (E max ), total length error (E l ) and mean stochastic error distance (SED) [53], [54].…”

Connections of Gini, Fisher, and Shannon by Bayes risk under proportional hazards

“…The MRL of X α , denoted as m α (τ ), is given by (3.2) with X α in place of X. Under the quadratic loss function 2 , X α > τ, the MRL m α (τ ) is the optimal decision for prediction of the excess:…”