A boosted decision tree approach using Bayesian hyper-parameter optimization for credit scoring

Xia, Yufei; Liu, Chuanzhe; Li, Yuying; Liu, Nana

doi:10.1016/j.eswa.2017.02.017

Cited by 589 publications

(307 citation statements)

References 49 publications

Supporting

Mentioning

267

Contrasting

Unclassified

Order By: Relevance

“…It is an advanced implementation of the gradient boosting (GB) algorithm and uses the decision tree as the base classifier. After carefully reading the research in [12] and [20], the algorithm of GB and XGBoost is briefly summarized as follows. Suppose we have a dataset D = { ; } containing n observations, where and denotes the features and the target variable, respectively.…”

Section: Extreme Gradient Boosting Along With Hyper-parametersmentioning

confidence: 99%

See 1 more Smart Citation

A Xgboost Risk Model via Feature Selection and Bayesian Hyper-Parameter Optimization

Wang¹,

Ni²

2019

IJDMS

View full text Add to dashboard Cite

This paper aims to explore models based on the extreme gradient boosting (XGBoost) approach for business risk classification. Feature selection (FS) algorithms and hyper-parameter optimizations are simultaneously considered during model training. The five most commonly used FS methods including weight by Gini, weight by Chi-square, hierarchical variable clustering, weight by correlation, and weight by information are applied to alleviate the effect of redundant features. Two hyper-parameter optimization approaches, random search (RS) and Bayesian tree-structured Parzen Estimator (TPE), are applied in XGBoost. The effect of different FS and hyper-parameter optimization methods on the model performance are investigated by the Wilcoxon Signed Rank Test. The performance of XGBoost is compared to the traditionally utilized logistic regression (LR) model in terms of classification accuracy, area under the curve (AUC), recall, and F1 score obtained from the 10-fold cross validation. Results show that hierarchical clustering is the optimal FS method for LR while weight by Chi-square achieves the best performance in XG-Boost. Both TPE and RS optimization in XGBoost outperform LR significantly. TPE optimization shows a superiority over RS since it results in a significantly higher accuracy and a marginally higher AUC, recall and F1 score. Furthermore, XGBoost with TPE tuning shows a lower variability than the RS method. Finally, the ranking of feature importance based on XGBoost enhances the model interpretation. Therefore, XGBoost with Bayesian TPE hyper-parameter optimization serves as an operative while powerful approach for business risk modeling.

show abstract

Section: Extreme Gradient Boosting Along With Hyper-parametersmentioning

confidence: 99%

“…It is a novel while advanced variant of the gradient boosting algorithm and has obtained promising results in many Kaggle machine learning competitions [10]. Furthermore, XGBoost has been successfully applied in bankruptcy prediction and credit scoring in a few studies [11] [12].…”

Section: Introductionmentioning

confidence: 99%

A Xgboost Risk Model via Feature Selection and Bayesian Hyper-Parameter Optimization

Wang¹,

Ni²

2019

IJDMS

View full text Add to dashboard Cite

show abstract

“…(2016)), neural networks (see West (2000)), gradient boosting (GBT, see Xia et al (2017)) and logistic regression. We use the out of sample loss as our main comparison metric, with lower loss values corresponding to better model performance.…”

Section: Model Comparisonsmentioning

confidence: 99%

Predicting Consumer Default: A Deep Learning Approach

Albanesi¹,

Vamossy²

2019

View full text Add to dashboard Cite

We develop a model to predict consumer default based on deep learning. We show that the model consistently outperforms standard credit scoring models, even though it uses the same data. Our model is interpretable and is able to provide a score to a larger class of borrowers relative to standard credit scoring models while accurately tracking variations in systemic risk. We argue that these properties can provide valuable insights for the design of policies targeted at reducing consumer default and alleviating its burden on borrowers and lenders, as well as macroprudential regulation. JEL Codes: C45; D14; E27; E44; G21; G24.in the United States. They are proprietary measures designed to rank borrowers based on their probability of future default. Specifically, they target the probability of a 90 days past due delinquency in the next 24 months. 1 Despite their ubiquitous use in the financial industry, there is very little information on credit scores, and emerging evidence suggests that as currently formulated credit scores have severe limitations. For example, Albanesi, De Giorgi, and Nosal (2017) show that during the 2007-2009 housing crisis there was a marked rise in mortgage delinquencies and foreclosures among high credit score borrowers, suggesting that credit scoring models at the time did not accurately reflect the probability of default for these borrowers. Additionally, it is well known that credit scores and indiscriminately low for young borrowers, and a substantial fraction of borrowers are unscored, which prevents them from accessing conventional forms of consumer credit.The Fair Credit Reporting Act, a legislation passed in 1970, and the Equal Opportunity in Credit Access Act of 1984 regulate credit scores and in particular determine which information can be included and must be excluded in credit scoring models. Such models can incorporate information in a borrower's credit report, except age and location. These restrictions are intended to prevent discrimination by age and factors related to location, such as race. 2 The law also mandates that entities that provide credit scores make public the four most important factors affecting scores. In marketing information, these are reported to be payment history, which is stated to explain about 35% of variation in credit scores, followed by amounts owed, length of credit history, new credit and credit mix, explaining 30%, 15%, 10% and 10% of the variation in credit scores respectively. Other than this, there is very little public information on credit scoring models, though several services are now available that allow consumers to simulate how various scenarios, such as paying off balances or taking out new loans, will affect their scores.The purpose of our analysis is to propose a model to predict consumer default that uses the same data as conventional credit scoring models, improves on their performance, benefiting both lenders and borrowers, and provides more transparency and accountability. To do so, we resort to deep learning, a type of machine l...

show abstract

“…Fahmi et al [55] and West and Bhattacharya [56] both used the same dataset, which is the one used in "UCSD- [64] proposed a sequential ensemble credit scoring model based on XGBoost, a variant of gradient boosting machine, and the hyperparameters of XGBoost are adaptively tuned by the tree-structured Parzen estimator, grid search, random search and manual search. A sequential ensemble learning combines a series of weak base learners that process different hypothesizes sequentially to form a better hypothesis, thus making good predictions [65] [66] [67].…”

Section: Credit Card Fraud Detectionmentioning

confidence: 99%

A state of the art survey of data mining-based fraud detection and credit scoring

Zhou¹,

Cheng²,

Zhu³

et al. 2018

MATEC Web Conf.

View full text Add to dashboard Cite

Abstract.Credit risk has been a widespread and deep penetrating problem for centuries, but not until various credit derivatives and products were developed and novel technologies began radically changing the human society, have fraud detection, credit scoring and other risk management systems become so important not only to some specific firms, but to industries and governments worldwide. Frauds and unpredictable defaults cost billions of dollars each year, thus, forcing financial institutions to continuously improve their systems for loss reduction. In the past twenty years, amounts of studies have proposed the use of data mining techniques to detect frauds, score credits and manage risks, but issues such as data selection, algorithm design, and hyperparameter optimization affect the perceived ability of the proposed solutions and it is difficult for auditors and researchers to explore and figure out the highest level of general development in this area. In this survey we focus on a state of the art survey of recently developed data mining techniques for fraud detection and credit scoring. Several outstanding experiments are recorded and highlighted, and the corresponding techniques, which are mostly based on supervised learning algorithms, unsupervised learning algorithms, semisupervised algorithms, ensemble learning, transfer learning, or some hybrid ideas are explained and analysed. The goal of this paper is to provide a dense review of up-to-date techniques for fraud detection and credit scoring, a general analysis on the results achieved and upcoming challenges for further researches.

show abstract

A boosted decision tree approach using Bayesian hyper-parameter optimization for credit scoring

Cited by 589 publications

References 49 publications

A Xgboost Risk Model via Feature Selection and Bayesian Hyper-Parameter Optimization

A Xgboost Risk Model via Feature Selection and Bayesian Hyper-Parameter Optimization

Predicting Consumer Default: A Deep Learning Approach

A state of the art survey of data mining-based fraud detection and credit scoring

Contact Info

Product

Resources

About