Head-to-head comparison of 14 prediction models for postoperative delirium in elderly non-ICU patients: an external validation study

Wong, Chung Kwan; Munster, Barbara C. Van; Hatseras, Athanasios; Veld, Else Huis In 't; Leeuwen, Barbara L. van; Rooij, Sophia E. de; Pleijhuis, Rick G.

doi:10.1136/bmjopen-2021-054023

Cited by 6 publications

(16 citation statements)

References 48 publications

(67 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Table 1 presents characteristics of the included studies. Prognostic models of hospital-induced delirium among adult medical-surgical patients were developed across a variety of countries, including the United States (n = 5), [ 8 , 23 , 24 , 27 , 30 ] the Netherlands (n = 3), [ 26 , 31 , 34 ] Austria (n = 1), [ 32 ] Chile (n = 1), [ 29 ] China (n = 1), [ 33 ] Japan (n = 1), [ 28 ] and the United Kingdom (n = 1) [ 25 ] between 1993 and 2022. Most of the studies were prospective cohorts (n = 10).…”

Section: Resultsmentioning

confidence: 99%

Risk of bias in prognostic models of hospital-induced delirium for medical-surgical units: A systematic review

Snigurska,

Liu,

Ser

et al. 2023

PLoS ONE

View full text Add to dashboard Cite

Purpose The purpose of this systematic review was to assess risk of bias in existing prognostic models of hospital-induced delirium for medical-surgical units. Methods APA PsycInfo, CINAHL, MEDLINE, and Web of Science Core Collection were searched on July 8, 2022, to identify original studies which developed and validated prognostic models of hospital-induced delirium for adult patients who were hospitalized in medical-surgical units. The Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies was used for data extraction. The Prediction Model Risk of Bias Assessment Tool was used to assess risk of bias. Risk of bias was assessed across four domains: participants, predictors, outcome, and analysis. Results Thirteen studies were included in the qualitative synthesis, including ten model development and validation studies and three model validation only studies. The methods in all of the studies were rated to be at high overall risk of bias. The methods of statistical analysis were the greatest source of bias. External validity of models in the included studies was tested at low levels of transportability. Conclusions Our findings highlight the ongoing scientific challenge of developing a valid prognostic model of hospital-induced delirium for medical-surgical units to tailor preventive interventions to patients who are at high risk of this iatrogenic condition. With limited knowledge about generalizable prognosis of hospital-induced delirium in medical-surgical units, existing prognostic models should be used with caution when creating clinical practice policies. Future research protocols must include robust study designs which take into account the perspectives of clinicians to identify and validate risk factors of hospital-induced delirium for accurate and generalizable prognosis in medical-surgical units.

show abstract

Section: Resultsmentioning

confidence: 99%

Risk of bias in prognostic models of hospital-induced delirium for medical-surgical units: A systematic review

Snigurska,

Liu,

Ser

et al. 2023

PLoS ONE

View full text Add to dashboard Cite

show abstract

“…This mainly affects the three MoCA subscores, which would require the entire MoCA to be conducted. A comprehensive way to assess these subscores was proposed by Wong and colleagues [ 43 ] in a modified version of the MoCA, which requires ~5 min. With the exception of the falls anamnesis, clinical frailty scale and the MoCA 5-min assessment [ 43 ], all features are routine data that could be extracted from the patient’s electronic medical record using an automated pipeline, the development of which is part of the SURGE-Ahead project [ 11 ].…”

Section: Discussionmentioning

confidence: 99%

“…A comprehensive way to assess these subscores was proposed by Wong and colleagues [ 43 ] in a modified version of the MoCA, which requires ~5 min. With the exception of the falls anamnesis, clinical frailty scale and the MoCA 5-min assessment [ 43 ], all features are routine data that could be extracted from the patient’s electronic medical record using an automated pipeline, the development of which is part of the SURGE-Ahead project [ 11 ]. In addition, the proposed POD prediction algorithm is capable of handling missing data by simple using the median/mean of the training data (see Table 2 ), albeit at the cost of accuracy.…”

Section: Discussionmentioning

confidence: 99%

Introducing a machine learning algorithm for delirium prediction—the Supporting SURgery with GEriatric Co-Management and AI project (SURGE-Ahead)

Benovic,

Ajlani,

Leinert

et al. 2024

Age and Ageing

View full text Add to dashboard Cite

Introduction Post-operative delirium (POD) is a common complication in older patients, with an incidence of 14–56%. To implement preventative procedures, it is necessary to identify patients at risk for POD. In the present study, we aimed to develop a machine learning (ML) model for POD prediction in older patients, in close cooperation with the PAWEL (patient safety, cost-effectiveness and quality of life in elective surgery) project. Methods The model was trained on the PAWEL study’s dataset of 878 patients (no intervention, age ≥ 70, 209 with POD). Presence of POD was determined by the Confusion Assessment Method and a chart review. We selected 15 features based on domain knowledge, ethical considerations and a recursive feature elimination. A logistic regression and a linear support vector machine (SVM) were trained, and evaluated using receiver operator characteristics (ROC). Results The selected features were American Society of Anesthesiologists score, multimorbidity, cut-to-suture time, estimated glomerular filtration rate, polypharmacy, use of cardio-pulmonary bypass, the Montreal cognitive assessment subscores ‘memory’, ‘orientation’ and ‘verbal fluency’, pre-existing dementia, clinical frailty scale, age, recent falls, post-operative isolation and pre-operative benzodiazepines. The linear SVM performed best, with an ROC area under the curve of 0.82 [95% CI 0.78–0.85] in the training set, 0.81 [95% CI 0.71–0.88] in the test set and 0.76 [95% CI 0.71–0.79] in a cross-centre validation. Conclusion We present a clinically useful and explainable ML model for POD prediction. The model will be deployed in the Supporting SURgery with GEriatric Co-Management and AI project.

show abstract

“…Many are based on retrospective data from conventional hospital records and therefore have an inherent bias from the underdiagnosis of POD (reviewed in [ 12 ]). Our model was developed with specialist clinical advice and in accordance with the recommendations of Lindroth et al [ 12 ], and it performs well in comparison with earlier prediction models [ 32 ]. Furthermore, all variables in the model have been independently confirmed as risk factors for delirium (reviewed in [ 33 ]).…”

Section: Discussionmentioning

confidence: 99%

Development and validation of an international preoperative risk assessment model for postoperative delirium

Dodsworth,

Reeve,

Falco

et al. 2023

Age and Ageing

View full text Add to dashboard Cite

Background Postoperative delirium (POD) is a frequent complication in older adults, characterised by disturbances in attention, awareness and cognition, and associated with prolonged hospitalisation, poor functional recovery, cognitive decline, long-term dementia and increased mortality. Early identification of patients at risk of POD can considerably aid prevention. Methods We have developed a preoperative POD risk prediction algorithm using data from eight studies identified during a systematic review and providing individual-level data. Ten-fold cross-validation was used for predictor selection and internal validation of the final penalised logistic regression model. The external validation used data from university hospitals in Switzerland and Germany. Results Development included 2,250 surgical (excluding cardiac and intracranial) patients 60 years of age or older, 444 of whom developed POD. The final model included age, body mass index, American Society of Anaesthesiologists (ASA) score, history of delirium, cognitive impairment, medications, optional C-reactive protein (CRP), surgical risk and whether the operation is a laparotomy/thoracotomy. At internal validation, the algorithm had an AUC of 0.80 (95% CI: 0.77–0.82) with CRP and 0.79 (95% CI: 0.77–0.82) without CRP. The external validation consisted of 359 patients, 87 of whom developed POD. The external validation yielded an AUC of 0.74 (95% CI: 0.68–0.80). Conclusions The algorithm is named PIPRA (Pre-Interventional Preventive Risk Assessment), has European conformity (ce) certification, is available at http://pipra.ch/ and is accepted for clinical use. It can be used to optimise patient care and prioritise interventions for vulnerable patients and presents an effective way to implement POD prevention strategies in clinical practice.

show abstract

Head-to-head comparison of 14 prediction models for postoperative delirium in elderly non-ICU patients: an external validation study

Cited by 6 publications

References 48 publications

Risk of bias in prognostic models of hospital-induced delirium for medical-surgical units: A systematic review

Risk of bias in prognostic models of hospital-induced delirium for medical-surgical units: A systematic review

Introducing a machine learning algorithm for delirium prediction—the Supporting SURgery with GEriatric Co-Management and AI project (SURGE-Ahead)

Development and validation of an international preoperative risk assessment model for postoperative delirium

Contact Info

Product

Resources

About