Random forest versus logit models: Which offers better early warning of fiscal stress?

This paper compares several methods for constructing weekly nowcasts of recession probabilities in Italy, with a focus on the most recent period of the Covid‐19 pandemic. The common thread of these methods is that they use, in different ways, the information content provided by financial market data. In particular, a battery of probit models are estimated after extracting information from a large dataset of more than 130 financial market variables observed at a weekly frequency. The accuracy of these models is explored in a pseudo out‐of‐sample nowcasting exercise. The results demonstrate that nowcasts derived from probit models estimated on a large set of financial variables are, on average, more accurate than those delivered by standard probit models estimated on a single financial covariate, such as the slope of the yield curve. The proposed approach performs well even compared with probit models estimated on single time series of real economic activity variables, such as industrial production, business tendency survey data or composite PMI indicators. Overall, the financial indicators used in this paper can be easily updated as soon as new data become available on a weekly basis, thus providing reliable early estimates of the Italian business cycle.

“…An alternative approach relies on a different functional form for the cumulative distribution function

normalΦ false(\cdot false)

in Equation (2), specifically the logistic function instead of the standard normal distribution, that is,

normalΦ false(s false) = \frac{e^{s}}{e^{s} + 1}

. Logit models have been often used in the literature on the effectiveness of early warning systems (Jarmulska, 2022) and therefore can serve as a benchmark to compare with.…”

Section: Empirical Analysismentioning

confidence: 99%

Nowcasting the state of the Italian economy: The role of financial markets

Ceci

Silvestrini

2023

“…Jabeur and Fahmi (2018) concluded that RF obtained the best classification result for firm failure prediction through a comparative study. Jarmulska (2022) compared the logit model and RF for the early warning of fiscal stress, and the results indicated that the RF model outperformed the logit model. In this sense, many studies have confirmed that RF has better prediction performance than other models in FDP.…”

Section: Literature Reviewmentioning

confidence: 99%

Interpreting the prediction results of the tree‐based gradient boosting models for financial distress prediction with an explainable machine learning approach

Liu

Ouyang

et al. 2022

Financial distress prediction is a major issue in the burgeoning fintech field. Given the importance of the reliability of the prediction results, there is an urgent need for the explanatory ability of the financial distress prediction model. From the modeling and explanation point of view, this study employs four prevailing tree‐based gradient boosting models, namely, gradient boosting decision tree, extreme gradient boosting, light gradient boosting machine, and categorical boosting, to build financial distress prediction models by using financial data of listed companies in China from 1998 to 2014 and five different prediction time spans. We observe that tree‐based gradient boosting models have better prediction performance than other prediction models. To explore the reasons for the prediction results, we deploy TreeSHAP. Then, we use Shapley regression to examine the statistically significant relationships between financial indicators and financial distress. We discover that financial indicators, such as net asset value per share and ratio of operating profits to current liabilities, are significantly related to financial distress. There is usually a nonlinear relationship between the financial predictors and prediction target. Thus, this study provides an effective method for financial distress prediction and an explanation of the results for listed companies in China.

“…Joy et al (2017) and Tanaka et al (2018) used the random forest to predict financial vulnerability. Tanaka et al (2016), Alessi and Detken (2018), and Jarmulska (2020) demonstrated that the random forest outperforms conventional methods in the early prediction of bank failures, banking crises, and fiscal stress, respectively. Onan and Toçoğlu (2020) compared five base learners with three widely utilized ensemble methods and showed that the random forest yields the best performance for satirical text identification in Turkish.…”

Section: Introductionmentioning

confidence: 99%

“…The essence of these approaches is to select the most applicable hyperparameters for these machine learning methods. To choose the optimal hyperparameters, recent studies have tended to focus on the out‐of‐sample predictive performance and have used k‐fold cross‐validation to determine the hyperparameters according to the out‐of‐sample predictive accuracy (Chetverikov et al, 2016; Hellwig, 2021; Holopainen & Sarlin, 2017; Jarmulska, 2020). Although simple cross‐validation of the training and test groups can alleviate the overfitting problem to a certain extent, this type of hyperparameter selection still inevitably refers to the information of the test group.…”

Section: Introductionmentioning

confidence: 99%

“…We are aware of only a few studies that have visualized these black boxes and performed a causal analysis. Jarmulska (2020) used Shapley values, partial dependence and accumulated local effects plots to explore the interpretability of the random forest. Strumbelj and Kononenko (2010), Lundberg and Lee (2017), and Tiffin (2019) used the Shapley value introduced by Shapley (1953) in cooperative game theory to explain machine learning methods.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Predicting financial crises with machine learning methods

Liu

Chen

Wang

2022

Countries must establish an effective early warning system to predict financial crises in order to avoid their catastrophic effects. To this end, we construct early warning systems based on the logistic model and seven machine learning methods, and we also use the Shapley value decomposition and Shapley regression to explore the causality of the machine learning methods. By comparing the performance of different early warning models in out-of-sample tests, we find that the machine learning models, especially the random forest, gradient boosting decision tree, and ensemble models, outperform the logistic model in terms of providing early predictions of financial crises. In addition, the Shapley value can be used to find more effective predictive indicators and analyze the causes of risks in different countries to a certain extent, enabling policymakers to supplement the policy toolbox to deal with such crises. Thus, we suggest that machine learning methods should be considered when establishing early warning systems to predict financial crises in the future.