Machine learning models for prediction of HF and CKD development in early-stage type 2 diabetes patients

Kanda, Eiichiro; Suzuki, Atsushi; Miyamoto, Masaki; Tsubota, Hiroo; Kanemata, Satomi; Shirakawa, Koichi; Yajima, Toshitaka

doi:10.1038/s41598-022-24562-2

Cited by 10 publications

(3 citation statements)

References 46 publications

(63 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We identified 74 studies that met our eligibility criteria. Of these, 66 studies presented the original version of the ML model, 10-75 and an additional eight studies externally validated these models. 76-…”

Section: Resultsmentioning

confidence: 99%

Machine Learning Models for Prediction of Diabetic Microvascular Complications

Kanbour,

Harris,

Lalani

et al. 2024

J Diabetes Sci Technol

View full text Add to dashboard Cite

Importance and Aims: Diabetic microvascular complications significantly impact morbidity and mortality. This review focuses on machine learning/artificial intelligence (ML/AI) in predicting diabetic retinopathy (DR), diabetic kidney disease (DKD), and diabetic neuropathy (DN). Methods: A comprehensive PubMed search from 1990 to 2023 identified studies on ML/AI models for diabetic microvascular complications. The review analyzed study design, cohorts, predictors, ML techniques, prediction horizon, and performance metrics. Results: Among the 74 identified studies, 256 featured internally validated ML models and 124 had externally validated models, with about half being retrospective. Since 2010, there has been a rise in the use of ML for predicting microvascular complications, mainly driven by DKD research across 27 countries. A more modest increase in ML research on DR and DN was observed, with publications from fewer countries. For all microvascular complications, predictive models achieved a mean (standard deviation) c-statistic of 0.79 (0.09) on internal validation and 0.72 (0.12) on external validation. Diabetic kidney disease models had the highest discrimination, with c-statistics of 0.81 (0.09) on internal validation and 0.74 (0.13) on external validation, respectively. Few studies externally validated prediction of DN. The prediction horizon, outcome definitions, number and type of predictors, and ML technique significantly influenced model performance. Conclusions and Relevance: There is growing global interest in using ML for predicting diabetic microvascular complications. Research on DKD is the most advanced in terms of publication volume and overall prediction performance. Both DR and DN require more research. External validation and adherence to recommended guidelines are crucial.

show abstract

Section: Resultsmentioning

confidence: 99%

Machine Learning Models for Prediction of Diabetic Microvascular Complications

Kanbour,

Harris,

Lalani

et al. 2024

J Diabetes Sci Technol

View full text Add to dashboard Cite

show abstract

“…Machine learning (ML) algorithms generate predictions from sample data without explicit directions from the user [1][2][3][4]. Common ML algorithms (e.g., XGBoost, Random Forest, Neural Networks) have been found to be more accurate than traditional parametric methods (linear regression, logistic regression) [5][6][7][8].…”

Section: Introductionmentioning

confidence: 99%

Increasing transparency in machine learning through bootstrap simulation and shapely additive explanations

Huang

2023

PLoS ONE

View full text Add to dashboard Cite

Machine learning methods are widely used within the medical field. However, the reliability and efficacy of these models is difficult to assess, making it difficult for researchers to identify which machine-learning model to apply to their dataset. We assessed whether variance calculations of model metrics (e.g., AUROC, Sensitivity, Specificity) through bootstrap simulation and SHapely Additive exPlanations (SHAP) could increase model transparency and improve model selection. Data from the England National Health Services Heart Disease Prediction Cohort was used. After comparison of model metrics for XGBoost, Random Forest, Artificial Neural Network, and Adaptive Boosting, XGBoost was used as the machine-learning model of choice in this study. Boost-strap simulation (N = 10,000) was used to empirically derive the distribution of model metrics and covariate Gain statistics. SHapely Additive exPlanations (SHAP) to provide explanations to machine-learning output and simulation to evaluate the variance of model accuracy metrics. For the XGBoost modeling method, we observed (through 10,000 completed simulations) that the AUROC ranged from 0.771 to 0.947, a difference of 0.176, the balanced accuracy ranged from 0.688 to 0.894, a 0.205 difference, the sensitivity ranged from 0.632 to 0.939, a 0.307 difference, and the specificity ranged from 0.595 to 0.944, a 0.394 difference. Among 10,000 simulations completed, we observed that the gain for Angina ranged from 0.225 to 0.456, a difference of 0.231, for Cholesterol ranged from 0.148 to 0.326, a difference of 0.178, for maximum heart rate (MaxHR) ranged from 0.081 to 0.200, a range of 0.119, and for Age ranged from 0.059 to 0.157, difference of 0.098. Use of simulations to empirically evaluate the variability of model metrics and explanatory algorithms to observe if covariates match the literature are necessary for increased transparency, reliability, and utility of machine learning methods. These variance statistics, combined with model accuracy statistics can help researchers identify the best model for a given dataset.

show abstract

“…Many risk models do currently exist for predicting risk of HF in patients with T2DM. [7][8][9][10] However, the majority of these models have been derived from studies that may not reflect real-world population (clinical trial data or prospective studies). For example, the WATCH-DM (Weight [BMI], Age, hyperTension, Creatinine, HDL-C, Diabetes control [fasting plasma glucose], QRS Duration, MI, and CABG) risk score and Risk Equations for Complications of Type 2 Diabetes risk scores were developed using the ACCORD trial (Action to Control Cardiovascular Disease in Diabetes), while the Thrombolysis in Myocardial Infarction Risk Score for Heart Failure in Diabetes (TRS-HF DM ) was created using the SAVOR-TIMI (Saxagliptin Assessment of Vascular Outcomes Recorded in Patients With Diabetes-Thrombolysis in Myocardial Infarction) 53 study.…”

mentioning

confidence: 99%

Impact of Residential Social Deprivation on Prediction of Heart Failure in Patients With Type 2 Diabetes: External Validation and Recalibration of the WATCH-DM Score Using Real World Data

Deo,

Al-Kindi,

Motairek

et al. 2024

Circ: Cardiovascular Quality and Outcomes

View full text Add to dashboard Cite

Background: Patients with type 2 diabetes are at risk of heart failure hospitalization. As social determinants of health are rarely included in risk models, we validated and recalibrated the WATCH-DM score in a diverse patient-group using their social deprivation index (SDI). Methods: We identified US Veterans with type 2 diabetes without heart failure that received outpatient care during 2010 at Veterans Affairs medical centers nationwide, linked them to their SDI using residential ZIP codes and grouped them as SDI <20%, 21% to 40%, 41% to 60%, 61% to 80%, and >80% (higher values represent increased deprivation). Accounting for all-cause mortality, we obtained the incidence for heart failure hospitalization at 5 years follow-up; overall and in each SDI group. We evaluated the WATCH-DM score using the C statistic, the Greenwood Nam D’Agostino test χ 2 test and calibration plots and further recalibrated the WATCH-DM score for each SDI group using a statistical correction factor. Results: In 1 065 691 studied patients (mean age 67 years, 25% Black and 6% Hispanic patients), the 5-year incidence of heart failure hospitalization was 5.39%. In SDI group 1 (least deprived) and 5 (most deprived), the 5-year heart failure hospitalization was 3.18% and 11%, respectively. The score C statistic was 0.62; WATCH-DM systematically overestimated heart failure risk in SDI groups 1 to 2 (expected/observed ratios, 1.38 and 1.36, respectively) and underestimated the heart failure risk in groups 4 to 5 (expected/observed ratios, 0.95 and 0.80, respectively). Graphical evaluation demonstrated that the recalibration of WATCH-DM using an SDI group-based correction factor improved predictive capabilities as supported by reduction in the χ 2 test results (801–27 in SDI groups I; 623–23 in SDI group V). Conclusions: Including social determinants of health to recalibrate the WATCH-DM score improved risk prediction highlighting the importance of including social determinants in future clinical risk prediction models.

show abstract

Machine learning models for prediction of HF and CKD development in early-stage type 2 diabetes patients

Cited by 10 publications

References 46 publications

Machine Learning Models for Prediction of Diabetic Microvascular Complications

Machine Learning Models for Prediction of Diabetic Microvascular Complications

Increasing transparency in machine learning through bootstrap simulation and shapely additive explanations

Impact of Residential Social Deprivation on Prediction of Heart Failure in Patients With Type 2 Diabetes: External Validation and Recalibration of the WATCH-DM Score Using Real World Data

Contact Info

Product

Resources

About