A systematic mapping study for ensemble classification methods in cardiovascular disease

Hosni, Mohamed; Gea, Juan M. Carrillo de; Idri, Ali; Bajta, Manal El; Alemán, José Luis Fernández; Garcı́a-Mateos, Ginés; Abnane, Ibtissam

doi:10.1007/s10462-020-09914-6

Cited by 13 publications

(8 citation statements)

References 118 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[38][39][40] The inclusion of high-performing and more diverse models such as Neural Network, and Xgboost may have contributed to the reduction of high variance and bias issues, which could potentially be detrimental to discrimination, calibration, clinical utility and overall accuracy across datasets and time. [41,42] For example, the logES-ESII-A ensemble substantially exceeded the performance reported in a small sized study that used an ensemble of GBM, RF, Support vector machine and Naïve Bayes, built using logES, ESII and other clinical variables without temporal consideration of variables, to predict cardiac postoperative mortality (AUC = 0.832 vs 0.795). [16] However, a smaller sized study that included Xgboost as part of a heterogeneous set of Super Learner ensemble did not achieve high performance using pre-operative data compared to this study's logES-ESII-A ensemble (AUC = 0.832 vs. 0.718 [0.687-0.749]), [43] or homogeneous Xgboost and RF (logES-ESII-P) ensembles (AUC = 0.832).…”

Section: Discussionmentioning

confidence: 98%

“…All models were evaluated using the applied Holdout dataset from the years 2017-2019. [41] Geometric average was used for all soft-voting transformations to bring probability distribution of base learners into one ensemble distribution. [59] Details of base learner model speci cation are provided in Supplementary Materials, Section 2.…”

Section: Ensemble Modellingmentioning

confidence: 99%

See 1 more Smart Citation

An Ensemble approach for Ensemble-Modelled Cardiac Surgery Risk Evaluation, Data Usage and Clinical Interpretability

Angelini

Dong

Sinha

et al. 2022

Preprint

View full text Add to dashboard Cite

Risk stratification plays a major role in the clinical decision-making process, patient consent and clinical governance analysis. However, the calibration of current risk scores (e.g., European System for Cardiac Operative Risk Evaluation (EuroSCORE), The Society of Thoracic Surgeons (STS) risk score) has been shown to deteriorate over time – a process known as calibration drift. The introduction of new clinical scores with different variable sets typically result in disparate datasets due to different levels of missingness. This is a barrier to the full insight and predictive capability of datasets across all potentially available time ranges. Little is known about the use of ensemble learning with ensemble metrics to mitigate the effects of calibration drift and changing risk across siloed datasets and time. In this study, we evaluated the effect of various combinations of Machine Learning (ML) models in improving model performance. The National Adult Cardiac Surgery Audit dataset was used (January 1996 to March 2019, 647,726 patients). We trained six different base learner models including Logistic Regression, Neuronetwork, Random Forest (RF), Weighted Support Vector Machine, Xgboost and Bayesian Update, based on two different variable sets of either Logistic EuroScore (LogES) or EuroScore II (ES II), partitioned by the time of score adoption (1996–2011 and 2012–2019). These base learner models are ensembled using nine different combinations to produce homogeneous or heterogeneous ensembles. Discrimination, calibration, clinical effectiveness and overall accuracy were assessed using an ensemble metric, referred to as clinical effectiveness metric (CEM). Xgboost homogenous ensemble (HE) was the highest performing model (CEM 0.725) with AUC (0.8327; 95% Confidence Interval (CI) 0.8323–0.8329) followed by Random Forest HE (CEM 0.723; AUC 0.8325; 95%CI 0.8320–0.8326). Across different heterogenous ensembles, significantly better performance was obtained by combining siloed datasets across time (CEM 0.720) than building ensembles of either 1996–2011 (t-test adjusted, p = 1.67e-6) or 2012–2019 (t-test adjusted, p = 1.35e-193) datasets alone. Both homogenous and heterogenous ML ensembles performed significantly better than traditional recalibration method (Bayesian Update). Combining the metrics covering all four aspects of discrimination, calibration, clinical usefulness and overall accuracy into a single ensemble metric improved the efficiency of cognitive decision-making. Xgboost/Random Forest homogenous ensembling and a highly heterogeneous ensemble approach showed high performance across multifaceted aspects of ML performance and were superior to traditional recalibration methods. Time-dependent ensemble combination of variables, having differing qualities according to time of score adoption, enabled previously siloed data to be combined, leading to increased power, clinical interpretability of variables and usage of data. For models to guide clinicians in individual decisions, performance exceeding these multifaceted benchmarks is necessary.

show abstract

Section: Discussionmentioning

confidence: 98%

Section: Ensemble Modellingmentioning

confidence: 99%

An Ensemble approach for Ensemble-Modelled Cardiac Surgery Risk Evaluation, Data Usage and Clinical Interpretability

Angelini

Dong

Sinha

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Instead, all models were evaluated using the Holdout dataset from the years 2017 to 2019 that were not part of the training process with performances compared to similar studies. 66 …”

Section: Methodsmentioning

confidence: 99%

Cardiac surgery risk prediction using ensemble machine learning to incorporate legacy risk scores: A benchmarking study

et al. 2023

View full text Add to dashboard Cite

Objective The introduction of new clinical risk scores (e.g. European System for Cardiac Operative Risk Evaluation (EuroSCORE) II) superseding original scores (e.g. EuroSCORE I) with different variable sets typically result in disparate datasets due to high levels of missingness for new score variables prior to time of adoption. Little is known about the use of ensemble learning to incorporate disparate data from legacy scores. We tested the hypothesised that Homogenenous and Heterogeneous Machine Learning (ML) ensembles will have better performance than ensembles of Dynamic Model Averaging (DMA) for combining knowledge from EuroSCORE I legacy data with EuroSCORE II data to predict cardiac surgery risk. Methods Using the National Adult Cardiac Surgery Audit dataset, we trained 12 different base learner models, based on two different variable sets from either EuroSCORE I (LogES) or EuroScore II (ES II), partitioned by the time of score adoption (1996–2016 or 2012–2016) and evaluated on holdout set (2017–2019). These base learner models were ensembled using nine different combinations of six ML algorithms to produce homogeneous or heterogeneous ensembles. Performance was assessed using a consensus metric. Results Xgboost homogenous ensemble (HE) was the highest performing model (clinical effectiveness metric (CEM) 0.725) with area under the curve (AUC) (0.8327; 95% confidence interval (CI) 0.8323–0.8329) followed by Random Forest HE (CEM 0.723; AUC 0.8325; 95%CI 0.8320–0.8326). Across different heterogenous ensembles, significantly better performance was obtained by combining siloed datasets across time (CEM 0.720) than building ensembles of either 1996–2011 ( t-test adjusted, p = 1.67×10−6) or 2012–2019 ( t-test adjusted, p = 1.35×10−193) datasets alone. Conclusions Both homogenous and heterogenous ML ensembles performed significantly better than DMA ensemble of Bayesian Update models. Time-dependent ensemble combination of variables, having differing qualities according to time of score adoption, enabled previously siloed data to be combined, leading to increased power, clinical interpretability of variables and usage of data.

show abstract

“…However, this result is not adequate and recognition of smaller class cases is much desirable than larger class cases. Class imbalance problem can be experienced in fraud detection [16], risk management [36,37], health care [38,39], software quality assurance [40][41][42], sentiment classification [43] and abstract classification [1], etc. The machine learning community has proposed various methods to address the requirements of class imbalance issue.…”

Section: Introductionmentioning

confidence: 99%

SMOTEMultiBoost: Leveraging the SMOTE with MultiBoost to Confront the Class Imbalance in Supervised Learning

Jhamat

Mustafa

Niu

2020

jictra

View full text Add to dashboard Cite

Class imbalance problem is being manifoldly confronted by researchers due to the increasing amount of complicated data. Common classification algorithms are impoverished to perform effectively on imbalanced datasets. Larger class cases typically outbalance smaller class cases in class imbalance learning. Common classification algorithms raise larger class performance owing to class imbalance in data and overall improvement in accuracy as their goal while lowering performance on smaller class. Furthermore, these algorithms deal false positive and false negative in an even way and regard equal cost of misclassifying cases. Meanwhile, different ensemble solutions have been proposed over the years for class imbalance learning but these approaches hamper the performance of larger class as emphasizing on the small class cases. The intuition of this overall degraded outcome would be the low diversity in ensemble solutions and overfitting or underfitting in data resampling techniques. To overcome these problems, we suggest a hybrid ensemble method by leveraging MultiBoost ensemble and Synthetic Minority Over-sampling TEchnique (SMOTE). Our suggested solution leverage the effectiveness of its elements. Therefore, it improves the outcome of the smaller class by reinforcing its space and limiting error in prediction. The proposed method shows improved performance as compare to numerous other algorithms and techniques in experiments.

show abstract

A systematic mapping study for ensemble classification methods in cardiovascular disease

Cited by 13 publications

References 118 publications

An Ensemble approach for Ensemble-Modelled Cardiac Surgery Risk Evaluation, Data Usage and Clinical Interpretability

An Ensemble approach for Ensemble-Modelled Cardiac Surgery Risk Evaluation, Data Usage and Clinical Interpretability

Cardiac surgery risk prediction using ensemble machine learning to incorporate legacy risk scores: A benchmarking study

SMOTEMultiBoost: Leveraging the SMOTE with MultiBoost to Confront the Class Imbalance in Supervised Learning

Contact Info

Product

Resources

About