Machine learning models predicting returns: Why most popular performance metrics are misleading and proposal for an efficient metric

Dessain, Jean

doi:10.1016/j.eswa.2022.116970

Cited by 28 publications

(8 citation statements)

References 146 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Assessing performance is a fundamental aspect of determining whether improvements are being achieved. The utilization of imbalanced data across classes may lead to accuracy scores that are satisfactory but ultimately misleading, as noted in reference [37] . The following metrics are utilized for the assessment of pipelines:…”

Section: Methodsmentioning

confidence: 99%

Classification of COVID-19 X-ray images using transfer learning with visual geometrical groups and novel sequential convolutional neural networks

Kumar¹,

Kumar²

2023

MethodsX

View full text Add to dashboard Cite

Section: Methodsmentioning

confidence: 99%

Classification of COVID-19 X-ray images using transfer learning with visual geometrical groups and novel sequential convolutional neural networks

Kumar¹,

Kumar²

2023

MethodsX

View full text Add to dashboard Cite

“…MSE is a popular metric for assessing the performance of MLP models [26], especially in regression tasks where continuous values are predicted. It measures the average squared difference between predicted and actual values.…”

Section: Model Evaluationmentioning

confidence: 99%

Novel Calibration Approach for Freeway Traffic Models in Single and Multi-Regimes Fundamental Diagram

Raufi,

maniat

2024

Preprint

View full text Add to dashboard Cite

The speed-density or flow-density relationship is a fundamental aspect of traffic flow theory. Single-regime models often face challenges in consistently fitting empirical data under diverse traffic conditions. This paper addresses the inaccuracies of single-regime models, attributing them not only to their functional forms but also to data dispersion and sample selection bias. To mitigate these issues, the data is segmented into day and night periods, leading to the creation of separate models for each. Calibration results, conducted on data from two freeways, Tehran-Karaj and Tehran-Qom, reveal distinct challenges, with the former exhibiting high dispersion and the latter showing low dispersion. Surprisingly, single-regime models demonstrate notable effectiveness when applied to distinct day and night data. Further analysis, categorizing data into day and night, results in reduced errors (20-60% improvement in R-Square), emphasizing the potential for enhanced accuracy by considering distinct regimes. Overall, the study underscores the significance of regime-specific considerations for accurate traffic flow modeling. Upon thorough evaluation, the study highlights the superiority of the neural network Multi-layer Perceptron (MLP) over traditional models. The research discusses the implications of incorporating day and night parameters into the models, emphasizing their potential for accuracy enhancement.

show abstract

“…The number of academic studies published on this topic has grown at an exponential rate and a comprehensive review of the literature is becoming increasingly challenging [1][2][3][4][5][6], if feasible at all. [7] offers the most comprehensive overview to date, with 190 articles reviewed over the period 2010 -June 2021, but with a narrow focus on the performance metrics used to compare algorithms predicting asset returns.…”

Section: Loss Functions In the Literaturementioning

confidence: 99%

Improving the Prediction of Asset Returns With Machine Learning by Using a Custom Loss Function

Dessain

2023

AAIML

View full text Add to dashboard Cite

Not all errors from models predicting asset returns are equal in terms of impact on the efficiency of the algorithm: a small error could trigger poor investment decisions while a significant error has no financial consequences. This economic asymmetry, critical for assessing the performance of algorithms, can usefully be replicated within the machine learning algorithms itself through the loss function to improve its prediction capability. . In this article: (a) we analyze symmetric and asymmetric loss functions for deep learning algorithms. We develop custom loss functions that mimic the asymmetry in economic consequences of prediction errors. (b) We compare the efficiency of these custom loss functions with MSE and the linearexponential loss “LinEx”. (c) We present an efficient custom loss function that significantly improves the prediction of asset returns with improved risk-return metrics (like Sharpe ratio twice better), and which we confirm to be robust.

show abstract

Machine learning models predicting returns: Why most popular performance metrics are misleading and proposal for an efficient metric

Cited by 28 publications

References 146 publications

Classification of COVID-19 X-ray images using transfer learning with visual geometrical groups and novel sequential convolutional neural networks

Classification of COVID-19 X-ray images using transfer learning with visual geometrical groups and novel sequential convolutional neural networks

Novel Calibration Approach for Freeway Traffic Models in Single and Multi-Regimes Fundamental Diagram

Improving the Prediction of Asset Returns With Machine Learning by Using a Custom Loss Function

Contact Info

Product

Resources

About