Assess and validate predictive performance of models for in-hospital mortality in COVID-19 patients: A retrospective cohort study in the Netherlands comparing the value of registry data with high-granular electronic health records

Vagliano, Iacopo; Schut, Martijn C; Abu‐Hanna, Ameen; Dongelmans, Dave A.; Lange, Dylan W. de; Gommers, Diederik; Cremer, Olaf L.; Bosman, Rob J.; Rigter, Sander; Wils, Evert‐Jan; Frenzel, Tim; Jong, Remko de; Peters, Marco; Kamps, Marlijn J. A.; Ramnarain, Dharmanand; Nowitzky, Ralph; Nooteboom, Fleur G. C. A.; Ruijter, Wouter de; Urlings‐Strop, Louise C.; Smit, Ellen G. M.; Mehagnoul-Schipper, D.J.; Dormans, Tom; Jager, Cornelis P.C. de; Hendriks, Stefaan H. A.; Achterberg, Sefanja; Oostdijk, Evelien; Reidinga, Auke C.; Festen-Spanjer, Barbara; Brunnekreef, Gert B.; Cornet, Alexander D.; Tempel, Walter van den; Boelens, Age D.; Koetsier, Peter; Lens, Judith; Faber, Harald J.; Karakus, Attila; Entjes, Robert; Jong, P. de; Rettig, Thijs C.D.; Reuland, M. C.; Arbous, Sesmu; Fleuren, Lucas M.; Dam, Tariq A.; Thoral, Patrick; Lalisang, Robbert C. A.; Tonutti, Michele; Bruin, Daan P. de; Elbers, Paul; Keizer, Nicolette F. de

doi:10.1016/j.ijmedinf.2022.104863

Cited by 5 publications

(3 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fourth, specific data from patient's electronic health records (EHRs), such as medication, was not available for the model development. However, adding more EHR data did not lead to better performance of priorly developed models predicting QoL and mortality (53,54). Future research with possible new modeling techniques and more widespread sample sizes should show whether adding more data to the model improves predictive power.…”

Section: Discussionmentioning

confidence: 93%

Prediction of Long-Term Physical, Mental, and Cognitive Problems Following Critical Illness: Development and External Validation of the PROSPECT Prediction Model*

van Sleeuwen,

Zegers,

Ramjith

et al. 2023

Critical Care Medicine

View full text Add to dashboard Cite

Objectives: ICU survivors often suffer from long-lasting physical, mental, and cognitive health problems after hospital discharge. As several interventions that treat or prevent these problems already start during ICU stay, patients at high risk should be identified early. This study aimed to develop a model for early prediction of post-ICU health problems within 48 hours after ICU admission. Design: Prospective cohort study in seven Dutch ICUs. Setting/Patients: ICU patients older than 16 years and admitted for greater than or equal to 12 hours between July 2016 and March 2020. Interventions: None. Measurements and Main Results: Outcomes were physical problems (fatigue or ≥ 3 new physical symptoms), mental problems (anxiety, depression, or post-traumatic stress disorder), and cognitive impairment. Patient record data and questionnaire data were collected at ICU admission, and after 3 and 12 months, of 2,476 patients. Several models predicting physical, mental, or cognitive problems and a composite score at 3 and 12 months were developed using variables collected within 48 hours after ICU admission. Based on performance and clinical feasibility, a model, PROSPECT, predicting post-ICU health problems at 3 months was chosen, including the predictors of chronic obstructive pulmonary disease, admission type, expected length of ICU stay greater than or equal to 2 days, and preadmission anxiety and fatigue. Internal validation using bootstrapping on data of the largest hospital (n = 1,244) yielded a C-statistic of 0.73 (95% CI, 0.70–0.76). External validation was performed on data (n = 864) from the other six hospitals with a C-statistic of 0.77 (95% CI, 0.73–0.80). Conclusions: The developed and externally validated PROSPECT model can be used within 48 hours after ICU admission for identifying patients with an increased risk of post-ICU problems 3 months after ICU admission. Timely preventive interventions starting during ICU admission and follow-up care can prevent or mitigate post-ICU problems in these high-risk patients.

show abstract

Section: Discussionmentioning

confidence: 93%

Prediction of Long-Term Physical, Mental, and Cognitive Problems Following Critical Illness: Development and External Validation of the PROSPECT Prediction Model*

van Sleeuwen,

Zegers,

Ramjith

et al. 2023

Critical Care Medicine

View full text Add to dashboard Cite

show abstract

“… The 0 h and 24 h logistic regression (LR) models are reported in two other articles. The LR model at admission (0 h) is included in Supplementary Table 4 of [ 9 ] and the LR model at 24 h is included in Supplementary Table s 2 of [ 10 ]. The probabilities of the regression tree are in good agreement with the means and medians of the predictions of the LR model.…”

Section: Resultsmentioning

confidence: 99%

Development and evaluation of regression tree models for predicting in-hospital mortality of a national registry of COVID-19 patients over six pandemic surges

Schut,

Dongelmans,

de Lange

et al. 2024

BMC Med Inform Decis Mak

View full text Add to dashboard Cite

Background Objective prognostic information is essential for good clinical decision making. In case of unknown diseases, scarcity of evidence and limited tacit knowledge prevent obtaining this information. Prediction models can be useful, but need to be not only evaluated on how well they predict, but also how stable these models are under fast changing circumstances with respect to development of the disease and the corresponding clinical response. This study aims to provide interpretable and actionable insights, particularly for clinicians. We developed and evaluated two regression tree predictive models for in-hospital mortality of COVID-19 patient at admission and 24 hours (24 h) after admission, using a national registry. We performed a retrospective analysis of observational routinely collected data. Methods Two regression tree models were developed for admission and 24 h after admission. The complexity of the trees was managed via cross validation to prevent overfitting. The predictive ability of the model was assessed via bootstrapping using the Area under the Receiver-Operating-Characteristic curve, Brier score and calibration curves. The tree models were assessed on the stability of their probabilities and predictive ability, on the selected variables, and compared to a full-fledged logistic regression model that uses variable selection and variable transformations using splines. Participants included COVID-19 patients from all ICUs participating in the Dutch National Intensive Care Evaluation (NICE) registry, who were admitted at the ICU between February 27, 2020, and November 23, 2021. From the NICE registry, we included concerned demographic data, minimum and maximum values of physiological data in the first 24 h of ICU admission and diagnoses (reason for admission as well as comorbidities) for model development. The main outcome measure was in-hospital mortality. We additionally analysed the Length-of-Stay (LoS) per patient subgroup per survival status. Results A total of 13,369 confirmed COVID-19 patients from 70 ICUs were included (with mortality rate of 28%). The optimism-corrected AUROC of the admission tree (with seven paths) was 0.72 (95% CI: 0.71–0.74) and of the 24 h tree (with 11 paths) was 0.74 (0.74–0.77). Both regression trees yielded good calibration and variable selection for both trees was stable. Patient subgroups comprising the tree paths had comparable survival probabilities as the full-fledged logistic regression model, survival probabilities were stable over six COVID-19 surges, and subgroups were shown to have added predictive value over the individual patient variables. Conclusions We developed and evaluated regression trees, which operate at par with a carefully crafted logistic regression model. The trees consist of homogenous subgroups of patients that are described by simple interpretable constraints on patient characteristics thereby facilitating shared decision-making.

show abstract

“…This study leveraged the rich dataset from the RUMC COVID-19 registry to analyze the association between age-related comorbid conditions and death across and within each wave of the COVID-19 pandemic since March 2020. While several studies have examined risk factors associated with hospitalizations, ICU admissions, and death, few have evaluated differences in these factors across distinct waves of the pandemic (18)(19)(20)(21)(22). The Alpha and Delta variants were associated with greater hospitalizations, ICU admissions, and mortality compared to the other variants.…”

Section: Discussionmentioning

confidence: 99%

COVID-19 waves in an urban setting 2020–2022: an electronic medical record analysis

Chen,

Gawel,

Desai

et al. 2024

Front. Public Health

View full text Add to dashboard Cite

BackgroundGlobal and national surveillance efforts have tracked COVID-19 incidence and clinical outcomes, but few studies have compared comorbid conditions and clinical outcomes across each wave of the pandemic. We analyzed data from the COVID-19 registry of a large urban healthcare system to determine the associations between presenting comorbidities and clinical outcomes during the pandemic.MethodsWe analyzed registry data for all inpatients and outpatients with COVID-19 from March 2020 through September 2022 (N = 44,499). Clinical outcomes were death, hospitalization, and intensive care unit (ICU) admission. Demographic and clinical outcomes data were analyzed overall and for each wave. Unadjusted and multivariable logistic regressions were performed to explore the associations between age, sex, race, ethnicity, comorbidities, and mortality.ResultsWaves 2 and 3 (Alpha and Delta variants) were associated with greater hospitalizations, ICU admissions, and mortality than other variants. Chronic pulmonary disease was the most common comorbid condition across all age groups and waves. Mortality rates were higher in older patients but decreased across all age groups in later waves. In every wave, mortality was associated with renal disease, congestive heart failure, cerebrovascular disease, diabetes, and chronic pulmonary disease. Multivariable analysis found that liver disease and renal disease were significantly associated with mortality, hospitalization, and ICU admission, and diabetes was significantly associated with hospitalization and ICU admission.ConclusionThe COVID-19 registry is a valuable resource to identify risk factors for clinical outcomes. Our findings may inform risk stratification and care planning for patients with COVID-19 based on age and comorbid conditions.

show abstract

Assess and validate predictive performance of models for in-hospital mortality in COVID-19 patients: A retrospective cohort study in the Netherlands comparing the value of registry data with high-granular electronic health records

Cited by 5 publications

References 37 publications

Prediction of Long-Term Physical, Mental, and Cognitive Problems Following Critical Illness: Development and External Validation of the PROSPECT Prediction Model*

Prediction of Long-Term Physical, Mental, and Cognitive Problems Following Critical Illness: Development and External Validation of the PROSPECT Prediction Model*

Development and evaluation of regression tree models for predicting in-hospital mortality of a national registry of COVID-19 patients over six pandemic surges

COVID-19 waves in an urban setting 2020–2022: an electronic medical record analysis

Contact Info

Product

Resources

About