Reviewing the use and quality of machine learning in developing clinical prediction models for cardiovascular disease

Allan, Simon; Olaiya, Raphael; Burhan, Rasan

doi:10.1136/postgradmedj-2020-139352

Cited by 14 publications

(9 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Established methods were used to develop and validate ESCC prediction models. 33 , 34 , 35 First, an initial analysis was based on participants with complete data. Then, multiple imputation with chained equations was used to replace the missing values of body mass index (BMI; calculated as weight in kilograms divided by height in meters squared [0.08%]), annual household income (0.02%), source of drinking water (4.42%), smoking status (0.04%), alcohol use status (0.03%), consumption of tea (0.03%), consumption of fresh fruit (1.83%), consumption of pickled food (1.08%), consumption of fried food (4.24%), consumption of hot food (3.78%), history of gastrointestinal tract diseases (0.04%), and family history of any cancer (0.05%).…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Development and Validation of Esophageal Squamous Cell Carcinoma Risk Prediction Models Based on an Endoscopic Screening Program

et al. 2023

View full text Add to dashboard Cite

ImportanceAssessment tools are lacking for screening of esophageal squamous cell cancer (ESCC) in China, especially for the follow-up stage. Risk prediction to optimize the screening procedure is urgently needed.ObjectiveTo develop and validate ESCC prediction models for identifying people at high risk for follow-up decision-making.Design, Setting, and ParticipantsThis open, prospective multicenter diagnostic study has been performed since September 1, 2006, in Shandong Province, China. This study used baseline and follow-up data until December 31, 2021. The data were analyzed between April 6 and May 31, 2022. Eligibility criteria consisted of rural residents aged 40 to 69 years who had no contraindications for endoscopy. Among 161 212 eligible participants, those diagnosed with cancer or who had cancer at baseline, did not complete the questionnaire, were younger than 40 years or older than 69 years, or were detected with severe dysplasia or worse lesions were eliminated from the analysis.ExposuresRisk factors obtained by questionnaire and endoscopy.Main Outcomes and MeasuresPathological diagnosis of ESCC and confirmation by cancer registry data.ResultsIn this diagnostic study of 104 129 participants (56.39% women; mean [SD] age, 54.31 [7.64] years), 59 481 (mean [SD] age, 53.83 [7.64] years; 58.55% women) formed the derivation set while 44 648 (mean [SD] age, 54.95 [7.60] years; 53.51% women) formed the validation set. A total of 252 new cases of ESCC were diagnosed during 424 903.50 person-years of follow-up in the derivation cohort and 61 new cases from 177 094.10 person-years follow-up in the validation cohort. Model A included the covariates age, sex, and number of lesions; model B included age, sex, smoking status, alcohol use status, body mass index, annual household income, history of gastrointestinal tract diseases, consumption of pickled food, number of lesions, distinct lesions, and mild or moderate dysplasia. The Harrell C statistic of model A was 0.80 (95% CI, 0.77-0.83) in the derivation set and 0.90 (95% CI, 0.87-0.93) in the validation set; the Harrell C statistic of model B was 0.83 (95% CI, 0.81-0.86) and 0.91 (95% CI, 0.88-0.95), respectively. The models also had good calibration performance and clinical usefulness.Conclusions and RelevanceThe findings of this diagnostic study suggest that the models developed are suitable for selecting high-risk populations for follow-up decision-making and optimizing the cancer screening process.

show abstract

Section: Methodsmentioning

confidence: 99%

“…The data were analyzed between April 6 and May 31, 2022. Established methods were used to develop and validate ESCC prediction models . First, an initial analysis was based on participants with complete data.…”

Section: Methodsmentioning

confidence: 99%

Development and Validation of Esophageal Squamous Cell Carcinoma Risk Prediction Models Based on an Endoscopic Screening Program

et al. 2023

View full text Add to dashboard Cite

show abstract

“…A meta-analysis on 344 studies showed that the SVM and GBM have the highest predictive ability [ 31 ]. A review article in 2022 indicated that RF and ANN have the best predictive performance [ 32 ]. So, in this study, the various supervised classical statistical and machine learning classification models were used by considering their predictive power and popularity, including Logistic Regression (LR), Linear Discriminant Analysis (LDA), Quadratic Discriminant Analysis (QDA), SVM, kNN, DT, RF, Bayesian Adaptive Regression Trees (BART), missing incorporated to attributes-within BART (BARTm), ANN and GBM.…”

Section: Methodsmentioning

confidence: 99%

Cardiovascular disease incidence prediction by machine learning and statistical techniques: a 16-year cohort study from eastern Mediterranean region

Mehrabani-Zeinabad

Feizi

Sadeghi

et al. 2023

BMC Med Inform Decis Mak

View full text Add to dashboard Cite

Background Cardiovascular diseases (CVD) are the predominant cause of early death worldwide. Identification of people with a high risk of being affected by CVD is consequential in CVD prevention. This study adopts Machine Learning (ML) and statistical techniques to develop classification models for predicting the future occurrence of CVD events in a large sample of Iranians. Methods We used multiple prediction models and ML techniques with different abilities to analyze the large dataset of 5432 healthy people at the beginning of entrance into the Isfahan Cohort Study (ICS) (1990–2017). Bayesian additive regression trees enhanced with “missingness incorporated in attributes” (BARTm) was run on the dataset with 515 variables (336 variables without and the remaining with up to 90% missing values). In the other used classification algorithms, variables with more than 10% missing values were excluded, and MissForest imputes the missing values of the remaining 49 variables. We used Recursive Feature Elimination (RFE) to select the most contributing variables. Random oversampling technique, recommended cut-point by precision-recall curve, and relevant evaluation metrics were used for handling unbalancing in the binary response variable. Results This study revealed that age, systolic blood pressure, fasting blood sugar, two-hour postprandial glucose, diabetes mellitus, history of heart disease, history of high blood pressure, and history of diabetes are the most contributing factors for predicting CVD incidence in the future. The main differences between the results of classification algorithms are due to the trade-off between sensitivity and specificity. Quadratic Discriminant Analysis (QDA) algorithm presents the highest accuracy (75.50 ± 0.08) but the minimum sensitivity (49.84 ± 0.25); In contrast, decision trees provide the lowest accuracy (51.95 ± 0.69) but the top sensitivity (82.52 ± 1.22). BARTm.90% resulted in 69.48 ± 0.28 accuracy and 54.00 ± 1.66 sensitivity without any preprocessing step. Conclusions This study confirmed that building a prediction model for CVD in each region is valuable for screening and primary prevention strategies in that specific region. Also, results showed that using conventional statistical models alongside ML algorithms makes it possible to take advantage of both techniques. Generally, QDA can accurately predict the future occurrence of CVD events with a fast (inference speed) and stable (confidence values) procedure. The combined ML and statistical algorithm of BARTm provide a flexible approach without any need for technical knowledge about assumptions and preprocessing steps of the prediction procedure.

show abstract

“…Alternatively, researchers comparing models to one another using discrimination measures may consider using models that have been developed in UK Biobank participants. Examples of the latter include machine learning models such as the elastic net-based Cox model by Agrawal et al 2021 Researchers may consider that the successful use of machine learning algorithms over traditional statistical approaches like QRISK3 depends on the transparency and standardisation of reporting, the testing and training of these algorithms on large scale databases, and formal independent external validations (Allan et al, 2021). The diversity of available machine learning algorithms also gives researchers a wide range of options, from which they may select a model that's well-aligned with the underlying dynamics of the data.…”

Section: Alternative Modelsmentioning

confidence: 99%

Clinical Prediction Models in Epidemiological Studies: Lessons from the Application of QRISK3 to UK Biobank Data

Parsons¹,

Colopy²,

Clifton³

et al. 2022

Journal of Data Science

View full text Add to dashboard Cite

Statistical models for clinical risk prediction are often derived using data from primary care databases; however, they are frequently used outside of clinical settings. The use of prediction models in epidemiological studies without external validation may lead to inaccurate results. We use the example of applying the QRISK3 model to data from the United Kingdom (UK) Biobank study to illustrate the challenges and provide suggestions for future authors. The QRISK3 model is recommended by the National Institute for Health and Care Excellence (NICE) as a tool to aid cardiovascular risk prediction in English and Welsh primary care patients aged between 40 and 74. QRISK3 has not been externally validated for use in studies where data is collected for more general scientific purposes, including the UK Biobank study. This lack of external validation is important as the QRISK3 scores of participants in UK Biobank have been used and reported in several publications. This paper outlines: (i) how various publications have used QRISK3 on UK Biobank data and (ii) the ways that the lack of external validation may affect the conclusions from these publications. We then propose potential solutions for addressing these challenges; for example, model recalibration and considering alternative models, for the application of traditional statistical models such as QRISK3, in cohorts without external validation.

show abstract

Reviewing the use and quality of machine learning in developing clinical prediction models for cardiovascular disease

Cited by 14 publications

References 38 publications

Development and Validation of Esophageal Squamous Cell Carcinoma Risk Prediction Models Based on an Endoscopic Screening Program

Development and Validation of Esophageal Squamous Cell Carcinoma Risk Prediction Models Based on an Endoscopic Screening Program

Cardiovascular disease incidence prediction by machine learning and statistical techniques: a 16-year cohort study from eastern Mediterranean region

Clinical Prediction Models in Epidemiological Studies: Lessons from the Application of QRISK3 to UK Biobank Data

Contact Info

Product

Resources

About