Development and validation of automated computer-aided risk scores to predict in-hospital mortality for emergency medical admissions with COVID-19: a retrospective cohort development and validation study

Abstract

ObjectivesThere are no established mortality risk equations specifically for unplanned emergency medical admissions which include patients with SARS-19 (COVID-19). We aim to develop and validate a computer-aided risk score (CARMc19) for predicting mortality risk by combining COVID-19 status, the first electronically recorded blood test results and the National Early Warning Score (NEWS2).DesignLogistic regression model development and validation study.SettingTwo acute hospitals (York Hospital—model development data; Scarborough Hospital—external validation data).ParticipantsAdult (aged ≥16 years) medical admissions discharged over a 24-month period with electronic NEWS and blood test results recorded on admission. We used logistic regression modelling to predict the risk of in-hospital mortality using two models: (1) CARMc19_N: age+sex+NEWS2 including subcomponents+COVID19; (2) CARMc19_NB: CARMc19_N in conjunction with seven blood test results and acute kidney injury score. Model performance was evaluated according to discrimination (c-statistic), calibration (graphically) and clinical usefulness at NEWS2 thresholds of 4+, 5+, 6+.ResultsThe risk of in-hospital mortality following emergency medical admission was similar in development and validation datasets (8.4% vs 8.2%). The c-statistics for predicting mortality for CARMc19_NB is better than CARMc19_N in the validation dataset (CARMc19_NB=0.88 (95% CI 0.86 to 0.90) vs CARMc19_N=0.86 (95% CI 0.83 to 0.88)). Both models had good calibration (CARMc19_NB=1.01 (95% CI 0.88 to 1.14) and CARMc19_N:0.95 (95% CI 0.83 to 1.06)). At all NEWS2 thresholds (4+, 5+, 6+) model, CARMc19_NB had better sensitivity and similar specificity.ConclusionsWe have developed a validated CARMc19 scores with good performance characteristics for predicting the risk of in-hospital mortality. Since the CARMc19 scores place no additional data collection burden on clinicians, it may now be carefully introduced and evaluated in hospitals with sufficient informatics infrastructure.

Accuracy of automated computer-aided risk scoring systems to estimate the risk of COVID-19: a retrospective cohort study

“…The CARS_N model is particularly attractive because it uses NEWS data which can be available within a short while (< 30 min) of admission and so can support There are several limitations to our study: (1) This data is from a single NHS Trust, and to understand the extent to which these findings are generalisable, further study is required (2) We used the index NEWS and blood test results which reflects the 'on-admission' risk of mortality of the patients. Nonetheless, NEWS and blood test results are repeatedly updated for each patient according to local hospital protocols (Figure S5 in supplementary material) (3) We identified COVID-19 based on ICD-10 code 'U071' which was determined by COVID-19 swab test results (hospital or community) and clinical judgment and so our findings are constrained by the accuracy of these methods [27,28] (4) We have used NEWS in our data but since the NEWS2 is now widely used, further study is required to determine the accuracy of NEWS2 based models [29].…”

Accuracy of automated computer-aided risk scoring systems to estimate the risk of COVID-19: a retrospective cohort study

“…There are several limitations to our study: (1) This is data from a single NHS Trust and the extent to which these ndings are generalisable, further study is required (2) We used the index NEWS and blood test results which re ects the 'on-admission' risk of mortality of the patients. Nonetheless, NEWS and blood test results are repeatedly updated for each patient according to local hospital protocols (Figure S5 in supplementary material) (3) We identi ed COVID-19 based on ICD-10 code 'U071' which was determined by COVID-19 swab test results (hospital or community) and clinical judgment and so our ndings are constrained by the accuracy of these methods [27,28] (4) We have used NEWS in our data but since the NEWS2 is now widely used, further study is required to determine the accuracy of NEWS2 based models [29].…”