Ensemble Learning for Disease Prediction: A Review

Abstract

Machine learning models are used to create and enhance various disease prediction frameworks. Ensemble learning is a machine learning technique that combines multiple classifiers to improve performance by making more accurate predictions than a single classifier. Although numerous studies have employed ensemble approaches for disease prediction, there is a lack of thorough assessment of commonly used ensemble approaches against highly researched diseases. Consequently, this study aims to identify significant trends in the performance accuracies of ensemble techniques (i.e., bagging, boosting, stacking, and voting) against five hugely researched diseases (i.e., diabetes, skin disease, kidney disease, liver disease, and heart conditions). Using a well-defined search strategy, we first identified 45 articles from the current literature that applied two or more of the four ensemble approaches to any of these five diseases and were published in 2016–2023. Although stacking has been used the fewest number of times (23) compared with bagging (41) and boosting (37), it showed the most accurate performance the most times (19 out of 23). The voting approach is the second-best ensemble approach, as revealed in this review. Stacking always revealed the most accurate performance in the reviewed articles for skin disease and diabetes. Bagging demonstrated the best performance for kidney disease (five out of six times) and boosting for liver and diabetes (four out of six times). The results show that stacking has demonstrated greater accuracy in disease prediction than the other three candidate algorithms. Our study also demonstrates variability in the perceived performance of different ensemble approaches against frequently used disease datasets. The findings of this work will assist researchers in better understanding current trends and hotspots in disease prediction models that employ ensemble learning, as well as in determining a more suitable ensemble model for predictive disease analytics. This article also discusses variability in the perceived performance of different ensemble approaches against frequently used disease datasets.

ESPDHot: An Effective Machine Learning-Based Approach for Predicting Protein–DNA Interaction Hotspots

“…Stacking, a well-known ensemble machine learning algorithm, combines multiple classification methods using a meta-model to output the prediction result. Compared with the single classification predictors, the stacking method has been demonstrated to be able to yield a much better performance in various prediction tasks . Here, our proposed method ESPDHot employs three single classifiers as the base classifiers: Support Vector Machine (SVM), eXtreme Gradient Boosting (XGBoost) and Artificial Neural Network (ANN), and the Logistic Regression (LR) predictor as the meta-classifier to combine the decisions from the single classifiers.…”

“…Still in 2020, Pan et al developed the PreHots approach where a sequential backward selection algorithm is utilized to select 19 optimal features for training an ensemble stacking classifier and obtained an MCC of 0.576 . Compared with the single classification predictors, the ensemble-based method has been demonstrated to be able to yield a much better performance in various prediction tasks such as lncRNA subcellular localizations, identification of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), prediction of potential B-cell epitopes for SARS-CoV-2, and the early kidney disease prediction . Apart from the above-mentioned classification models, the regression models can also be used to predict hotspots when given a ΔΔ G cutoff.…”

“…4 Compared with the single classification predictors, the ensemble-based method has been demonstrated to be able to yield a much better performance in various prediction tasks such as lncRNA subcellular localizations, 7 2), 8 prediction of potential B-cell epitopes for SARS-CoV-2, 9 and the early kidney disease prediction. 10 Apart from the above-mentioned classification models, the regression models can also be used to predict hotspots when given a ΔΔG cutoff.…”

Predictive modelling and identification of key risk factors for stroke using machine learning

“…By combining the insights from various models, the Voting ensemble aims to maximize the overall predictive power of the DSE model. It's like having a panel of experts vote on the most likely outcome, with the best expert's opinion carrying the most weight 45 .…”

Ensemble machine learning prediction of hyperuricemia based on a prospective health checkup population

“…There are three main types of ensemble learning algorithms: bagging, boosting, and stacking, each with its unique way of model combination ( Zhou, 2021 ). Stacking trains multiple first-level models with different algorithms on the same dataset and combines their predictions using a second-level model, known as the meta-learner, to produce one more accurate and robust prediction ( Mahajan et al, 2023 ). We aimed to use the stacking ensemble technique to build an accurate HUA risk prediction model, integrating the results of support vector machine (SVM), decision tree C5.0 (C5.0), and eXtreme gradient boosting (XGBoost) to improve the final performance.…”