Systematic Reviews of Prediction Models

Collins, Gary S.; Moons, Karel G.M.; Debray, Thomas P. A.; Altman, Douglas G.; Riley, Richard D

doi:10.1002/9781119099369.ch18

Cited by 4 publications

(8 citation statements)

References 118 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The meta-analysis reports the 95% prediction interval to account for varying model performance due to differences in case mix and other study level factors. 52 …”

Section: Resultsmentioning

confidence: 99%

Hospital length of stay prediction for general surgery and total knee arthroplasty admissions: Systematic review and meta-analysis of published prediction models

et al. 2023

View full text Add to dashboard Cite

Objective Systematic review of length of stay (LOS) prediction models to assess the study methods (including prediction variables), study quality, and performance of predictive models (using area under receiver operating curve (AUROC)) for general surgery populations and total knee arthroplasty (TKA). Method LOS prediction models published since 2010 were identified in five major research databases. The main outcomes were model performance metrics including AUROC, prediction variables, and level of validation. Risk of bias was assessed using the PROBAST checklist. Results Five general surgery studies (15 models) and 10 TKA studies (24 models) were identified. All general surgery and 20 TKA models used statistical approaches; 4 TKA models used machine learning approaches. Risk scores, diagnosis, and procedure types were predominant predictors used. Risk of bias was ranked as moderate in 3/15 and high in 12/15 studies. Discrimination measures were reported in 14/15 and calibration measures in 3/15 studies, with only 4/39 externally validated models (3 general surgery and 1 TKA). Meta-analysis of externally validated models (3 general surgery) suggested the AUROC 95% prediction interval is excellent and ranges between 0.803 and 0.970. Conclusion This is the first systematic review assessing quality of risk prediction models for prolonged LOS in general surgery and TKA groups. We showed that these risk prediction models were infrequently externally validated with poor study quality, typically related to poor reporting. Both machine learning and statistical modelling methods, plus the meta-analysis, showed acceptable to good predictive performance, which are encouraging. Moving forward, a focus on quality methods and external validation is needed before clinical application.

show abstract

“…The meta-analysis reports the 95% prediction interval to account for varying model performance due to differences in case mix and other study level factors. 52 …”

Section: Resultsmentioning

confidence: 99%

Hospital length of stay prediction for general surgery and total knee arthroplasty admissions: Systematic review and meta-analysis of published prediction models

et al. 2023

View full text Add to dashboard Cite

show abstract

“…The studies will likely show significant clinical, methodological and statistical heterogeneity. We will therefore synthesize data quantitatively in appropriate subgroups (prediction of CKD development, CKD diagnosis, and prediction of CKD progression to ESKD), guided by Collins et al 2022 –specific to the conduct of systematic reviews of prediction models, including the methods for quantitative synthesis [ 39 ].…”

Section: Methodsmentioning

confidence: 99%

“…Quantitative data synthesis. As recommended by Collins et al 2022 [39], random-effects meta-analysis will be used to summarize estimates of model discrimination and calibration.…”

Section: Data Synthesismentioning

confidence: 99%

Prediction and diagnosis of chronic kidney disease development and progression using machine-learning: Protocol for a systematic review and meta-analysis of reporting standards and model performance

et al. 2023

View full text Add to dashboard Cite

Chronic Kidney disease (CKD) is an important yet under-recognized contributor to morbidity and mortality globally. Machine-learning (ML) based decision support tools have been developed across many aspects of CKD care. Notably, algorithms developed in the prediction and diagnosis of CKD development and progression may help to facilitate early disease prevention, assist with early planning of renal replacement therapy, and offer potential clinical and economic benefits to patients and health systems. Clinical implementation can be affected by the uncertainty surrounding the methodological rigor and performance of ML-based models. This systematic review aims to evaluate the application of prognostic and diagnostic ML tools in CKD development and progression. The protocol has been prepared using the Preferred Items for Systematic Review and Meta-analysis Protocols (PRISMA-P) guidelines. The systematic review protocol for CKD prediction and diagnosis have been registered with the International Prospective Register of Systematic Reviews (PROSPERO) (CRD42022356704, CRD42022372378). A systematic search will be undertaken of PubMed, Embase, the Cochrane Central Register of Controlled Trials (CENTRAL), the Web of Science, and the IEEE Xplore digital library. Studies in which ML has been applied to predict and diagnose CKD development and progression will be included. The primary outcome will be the comparison of the performance of ML-based models with non-ML-based models. Secondary analysis will consist of model use cases, model construct, and model reporting quality. This systematic review will offer valuable insight into the performance and reporting quality of ML-based models in CKD diagnosis and prediction. This will inform clinicians and technical specialists of the current development of ML in CKD care, as well as direct future model development and standardization.

show abstract

“…We defined ‘diagnostic model’ as any statistical model predicting the risk or odds of inpatients developing HO-CDI ≥48 hours after admission. 9 We included studies of any design, language, and publication date before January 21, 2022, with any timeframe (eg, prospective, retrospective). We excluded studies that included pediatric patients (aged <18 years), community-associated CDI, performed solely univariate analysis, used control group of patients with diarrhea, or described diagnostic models for individuals with a diagnosis of CDI.…”

Section: Methodsmentioning

confidence: 99%

Diagnostic prediction models to identify patients at risk for healthcare-facility–onset Clostridioides difficile: A systematic review of methodology and reporting

Patterson,

Fajnzylber,

Nero

et al. 2023

Infect. Control Hosp. Epidemiol.

View full text Add to dashboard Cite

Objective: To systematically review the methodology, performance, and generalizability of diagnostic models for predicting the risk of healthcare-facility–onset (HO) Clostridioides difficile infection (CDI) in adult hospital inpatients (aged ≥18 years). Background: CDI is the most common cause of healthcare-associated diarrhea. Prediction models that identify inpatients at risk of HO-CDI have been published; however, the quality and utility of these models remain uncertain. Methods: Two independent reviewers evaluated articles describing the development and/or validation of multivariable HO-CDI diagnostic models in an inpatient setting. All publication dates, languages, and study designs were considered. Model details (eg, sample size and source, outcome, and performance) were extracted from the selected studies based on the CHARMS checklist. The risk of bias was further assessed using PROBAST. Results: Of the 3,030 records evaluated, 11 were eligible for final analysis, which described 12 diagnostic models. Most studies clearly identified the predictors and outcomes but did not report how missing data were handled. The most frequent predictors across all models were advanced age, receipt of high-risk antibiotics, history of hospitalization, and history of CDI. All studies reported the area under the receiver operating characteristic curve (AUROC) as a measure of discriminatory ability. However, only 3 studies reported the model calibration results, and only 2 studies were externally validated. All of the studies had a high risk of bias. Conclusion: The studies varied in their ability to predict the risk of HO-CDI. Future models will benefit from the validation on a prospective external cohort to maximize external validity.

show abstract

Systematic Reviews of Prediction Models

Cited by 4 publications

References 118 publications

Hospital length of stay prediction for general surgery and total knee arthroplasty admissions: Systematic review and meta-analysis of published prediction models

Hospital length of stay prediction for general surgery and total knee arthroplasty admissions: Systematic review and meta-analysis of published prediction models

Prediction and diagnosis of chronic kidney disease development and progression using machine-learning: Protocol for a systematic review and meta-analysis of reporting standards and model performance

Diagnostic prediction models to identify patients at risk for healthcare-facility–onset Clostridioides difficile: A systematic review of methodology and reporting

Contact Info

Product

Resources

About