Conditional Superior Predictive Ability

Jia, Lee; Liao, Zhipeng; Quaedvlieg, Rogier

doi:10.2139/ssrn.3536461

Cited by 10 publications

(18 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thus, we have the surprising result that while the mean is more robust than the median in forecast evaluation, the opposite is well-known to hold in classical estimation theory. for tests of equal conditional predictive ability and tests of superior conditional predictive ability (Li et al, 2021+). Second, similar to the property of strict consistency for loss functions , robustness should be considered with respect to a specified class P of distributions.…”

Section: Characterizing Robust Loss Functionsmentioning

confidence: 99%

On Testing Equal Conditional Predictive Ability Under Measurement Error

Hoga¹,

Dimitriadis²

2021

Preprint

View full text Add to dashboard Cite

Loss functions are widely used to compare several competing forecasts. However, forecast comparisons are often based on mismeasured proxy variables for the true target.We introduce the concept of exact robustness to measurement error for loss functions and fully characterize this class of loss functions as the Bregman class. For such exactly robust loss functions, forecast loss differences are on average unaffected by the use of proxy variables and, thus, inference on conditional predictive ability can be carried out as usual. Moreover, we show that more precise proxies give predictive ability tests higher power in discriminating between competing forecasts. Simulations illustrate the different behavior of exactly robust and non-robust loss functions. An empirical application to US GDP growth rates demonstrates that it is easier to discriminate between forecasts issued at different horizons if a better proxy for GDP growth is used.

show abstract

Section: Characterizing Robust Loss Functionsmentioning

confidence: 99%

On Testing Equal Conditional Predictive Ability Under Measurement Error

Hoga¹,

Dimitriadis²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…However, within our analysis, we notice that Hansen's (2005) test alone still fails to distinguish some prediction models' performance, which is also the case for the Diebold and Mariano (1995) test used in Gu et al (2020) . To address this issue, we further look into the models' conditional predictive ability using the conditional superior predictive ability (CSPA) test in Li et al (2020) , which allows us to compare the performance of machine learning methods in different macroeconomic environments. See Internet Appendix B for a detailed description of both tests.…”

Section: Alternative Model Selectionmentioning

confidence: 99%

“…The first column reports the number of rejections of the one-versus-one USPA test for row models at the 5% significance level based on the full sample. The next six columns report similar summary statistics of the conditional superior predictive ability tests ( Li et al (2020) ) for different conditioning variables. For the CSPA tests, the entries report the number of rejections of the CSPA tests against the rest 12 competing models for a specific pair of the row model and the column conditioning variable.…”

Section: Alternative Model Selectionmentioning

confidence: 99%

“…Moreover, comparing the out-of-sample R 2 with studies in the US market, the Chinese market reveals substantially more predictability. As the out-of-sample R 2 has some limitations for model selection, we analyze the models' conditional predictive ability using a statistical test developed in Li et al (2020) , which allows us to compare the performance of machine learning methods in different macroeconomic environments. Again, the neural networks prove robust to this new statistical test and emerge as the best-performing method in terms of predictability.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Machine-Learning in the Chinese Factor Zoo

2020

View full text Add to dashboard Cite

We add to the emerging literature on empirical asset pricing in the Chinese stock market by building and analyzing a comprehensive set of factors with 1,160 signals for return prediction. Using various machine learning algorithms, we investigate which signals dominate in the Chinese market, a market characterized by a large proportion of retail investors with speculative motives, state-owned firms, and short-sales restrictions. Contrary to studies for the U.S. market, liquidity and fundamental factors emerge as the most important predictors, while price trend signals are less significant. We find that retail investors' dominating presence positively affects short-term predictability, particularly for small stocks. Another feature that distinguishes the Chinese from the U.S. market is the high predictability of large stocks and state-owned enterprises over longer horizons. Our portfolio analysis shows that this overall increased predictability leads to significantly higher out-of-sample performance than in other markets, which remains economically significant after transaction costs.

show abstract

“…The Supplemental Appendix S6 contains a more thorough analysis of the forecast performance of the PV(G)-HAR model based on the Conditional Superior Predictive Ability (CSPA) test ofLi, Liao and Quaedvlieg (2020). As that analysis also shows, the improved performance can only be partly attributed to the model isolating jumps.…”

mentioning

confidence: 99%