New hybrid data mining model for credit scoring based on feature selection algorithm and ensemble classifiers

Nalić, Jasmina; Martinović, Goran; Žagar, Drago

doi:10.1016/j.aei.2020.101130

Cited by 54 publications

(17 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many existing ensemble models are changes or improvements of the two methods [ 8 , 9 ]. As part of data preprocessing, feature selection algorithms have been proven by many researchers to improve the performance of machine learning models [ 10 ].…”

Section: Introductionmentioning

confidence: 99%

A Two-Stage Hybrid Default Discriminant Model Based on Deep Forest

Liu

et al. 2021

Entropy

View full text Add to dashboard Cite

Background: the credit scoring model is an effective tool for banks and other financial institutions to distinguish potential default borrowers. The credit scoring model represented by machine learning methods such as deep learning performs well in terms of the accuracy of default discrimination, but the model itself also has many shortcomings such as many hyperparameters and large dependence on big data. There is still a lot of room to improve its interpretability and robustness. Methods: the deep forest or multi-Grained Cascade Forest (gcForest) is a decision tree depth model based on the random forest algorithm. Using multidimensional scanning and cascading processing, gcForest can effectively identify and process high-dimensional feature information. At the same time, gcForest has fewer hyperparameters and has strong robustness. So, this paper constructs a two-stage hybrid default discrimination model based on multiple feature selection methods and gcForest algorithm, and at the same time, it optimizes the parameters for the lowest type II error as the first principle, and the highest AUC and accuracy as the second and third principles. GcForest can not only reflect the advantages of traditional statistical models in terms of interpretability and robustness but also take into account the advantages of deep learning models in terms of accuracy. Results: the validity of the hybrid default discrimination model is verified by three real open credit data sets of Australian, Japanese, and German in the UCI database. Conclusion: the performance of the gcForest is better than the current popular single classifiers such as ANN, and the common ensemble classifiers such as LightGBM, and CNNs in type II error, AUC, and accuracy. Besides, in comparison with other similar research results, the robustness and effectiveness of this model are further verified.

show abstract

Section: Introductionmentioning

confidence: 99%

A Two-Stage Hybrid Default Discriminant Model Based on Deep Forest

Liu

et al. 2021

Entropy

View full text Add to dashboard Cite

show abstract

“…Soft voting provides better performance when compared to hard voting. The reason behind is that the soft voting method gives more extra weight to secure voting (Nalić, Martinović, & Žagar, 2020) and considers the precision of each classifier in the final decision. This research combines the output probabilities of a single sample for each combination of classifiers and makes the decision for each sample based on the average probability for the target class.…”

Section: Methodsmentioning

confidence: 99%

New hybrid data mining model for prediction of Salmonella presence in agricultural waters based on ensemble feature selection and machine learning algorithms

Buyrukoğlu

2021

Journal of Food Safety

View full text Add to dashboard Cite

This paper aims to create a new hybrid ensemble data mining model to predict the Salmonella presence in agricultural surface waters based on the combination of heterogeneous ensemble approach for feature selection, clustering, regression, and classification algorithms. The data set for this study was collected from six agricultural ponds in Central Florida consisting of 23 features with 540 instances (26 Salmonella positive and 514 Salmonella negative). The model consisted of three stages. Initially, a heterogeneous ensemble feature selection (HEFS) approach was applied to select top features. Then, the k‐means clustering algorithm was implemented to remove misclassified cases from the data set. Finally, classification and regression algorithms, including support vector machine (SVM), Naïve Bayes (NB), Artificial Neural Network (ANN), Random Forest (RF) with soft voting approach were applied to the preprocessed data set to predict the Salmonella presence in agricultural surface waters with the amount of test set (20%). These algorithms were combined in 10 different ensemble models through the soft voting approach. The performance of these hybrid ensemble models was also evaluated. The ensemble ANN + RF model achieved the highest performance and outperformed all other single and ensemble models based on Area under the ROC Curve (AUC) (0.98) and prediction accuracy (94.9%). The findings emphasize the validity of our hybrid ensemble model which encourages researchers to predict Salmonella presence in agricultural surface waters.

show abstract

“…They also recommend that researchers could explore different approaches, offer practitioners suggestions on selecting appropriate methods for usage in actual situations, and outline probable restrictions and concerns for future work. Nalić et al [76] suggested a hybrid EDM model (Generalized Linear Model (GLM) + DT that integrates several FS (e.g., Classifier Feature Evaluation (ClassFE), Correlation Feature Evaluator (CorrelationFE), Gain Ratio Feature Evaluator (GainRFE), Information Gain Feature Evaluator (InfoGainFE), Relief Feature Evaluator (RefilefFE)), and learning classification methods to help in decision making in the context of dimensionality reduction.…”

Section: Feature Selection (Fs) and Educational Data Mining (Edm)mentioning

confidence: 99%

An Efficient Data Mining Technique for Assessing Satisfaction Level With Online Learning for Higher Education Students During the COVID-19

et al. 2022

View full text Add to dashboard Cite

All the educational organizations mainly aim at elevating the academic performance of students for improving the overall quality of education. In this direction, Educational Data Mining (EDM) is a rapidly trending research area that utilizes the essence of Data Mining (DM) concepts to help academic institutions figure out useful information on the Student Satisfaction Level (SSL) with the Online Learning process (OL) during COVID-19 lock-down. Different practices have been tried with EDM to predict students' behaviors to reach the best educational settings. Therefore, Feature Selection (FS) is typically employed to find the most relevant subset of features with minimum cardinality. As the predictive accuracy of a satisfaction model is significantly influenced by the FS process, the effectiveness of the SSL model is elaborately studied in this paper in connection with FS techniques. In this connection, a dataset was first collected online via a questionnaire of student reviews on OL courses. Using this datatset, the performance of wrapper FS techniques in DM and classification algorithms was analyzed in terms of fitness values. Ultimately, the goodness of subsets with different cardinalities is evaluated in terms of prediction accuracy and number of selected features by measuring the quality of 11 wrapper-based FS algorithms and the k-Nearest Neighbor (k-NN) and Support Vector Machine (SVM) as base-line classifiers. Based on the experiments, the optimal dimensionality of the feature subset was revealed, as well as the best method. The findings of the present study evidently support the well-known conjunction of the existence of minimum number of features and an increase in predictive accuracy. It is remarkable the relevancy of FS for highaccuracy SSL prediction, as the relevant set of features can effectively assist in deriving constructive educational strategies. Our study contributes a feature size reduction of up to 80% along with up to 100% classification accuracy on the adopted real-time dataset.

show abstract

New hybrid data mining model for credit scoring based on feature selection algorithm and ensemble classifiers

Cited by 54 publications

References 14 publications

A Two-Stage Hybrid Default Discriminant Model Based on Deep Forest

A Two-Stage Hybrid Default Discriminant Model Based on Deep Forest

New hybrid data mining model for prediction of Salmonella presence in agricultural waters based on ensemble feature selection and machine learning algorithms

An Efficient Data Mining Technique for Assessing Satisfaction Level With Online Learning for Higher Education Students During the COVID-19

Contact Info

Product

Resources

About