A prospective study of consecutive emergency medical admissions to compare a novel automated computer-aided mortality risk score and clinical judgement of patient mortality risk

Faisal, Muhammad; Khatoon, Binish; Scally, Andy J.; Richardson, Donald; Irwin, Sally; Davidson, Rachel A.; Heseltine, D.; Corlett, Alison; Ali, Javed; Hampson, Rebecca; Kesavan, Sandeep; McGonigal, Gerry; Goodman, Karen J.; Harkness, Michael; Mohammed, Mohammed

doi:10.1136/bmjopen-2018-027741

Cited by 5 publications

(6 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Not surprisingly, CARM_NB is found to be good discrimination and calibration which is consistent with the literature [5,20] as CARM_NB was developed for predicting the in-hospital mortality. However, CARS_N was developed for predicting sepsis and we found it has good discrimination and calibration compared to other CARSS models.…”

Section: Discussionsupporting

confidence: 88%

Accuracy of automated computer aided-risk scoring systems to estimate the risk of COVID-19 and in-hospital mortality: a retrospective cohort study

Faisal

Mohammed

Richardson

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

ObjectivesAlthough a set of computer-aided risk scoring systems (CARSS), that use the National Early Warning Score and routine blood tests results, have been validated for predicting in-hospital mortality and sepsis in unplanned admission to hospital, little is known about their performance for COVID-19 patients. We compare the performance of CARSS in unplanned admissions with COVID-19 during the first phase of the pandemic.Designa retrospective cross-sectional studySettingTwo acute hospitals (Scarborough and York) are combined into a single dataset and analysed collectively.ParticipantsAdult (>=18 years) non-elective admissions discharged between 11-March-2020 to 13-June-2020 with an index NEWS electronically recorded within ±24 hours. We assessed the performance of all four risk score (for sepsis: CARS_N, CARS_NB; for mortality: CARM_N, CARM_NB) according to discrimination (c-statistic) and calibration (graphically) in predicting the risk of COVID-19 and in-hospital mortality.ResultsThe risk of in-hospital mortality following emergency medical admission was 8.4% (500/6444) and 9.6% (620/6444) had a diagnosis of COVID-19. For predicting COVID-19 admissions, the CARS_N model had the highest discrimination 0.73 (0.71 to 0.75) and calibration slope 0.81 (0.72 to 0.89). For predicting in-hospital mortality, the CARM_NB model had the highest discrimination 0.84 (0.82 to 0.75) and calibration slope 0.89 (0.81 to 0.98).ConclusionsTwo of the computer-aided risk scores (CARS_N and CARM_NB) are reasonably accurate for predicting the risk of COVID-19 and in-hospital mortality, respectively. They may be clinically useful as an early warning system at the time of admission especially to triage large numbers of unplanned hospital admissions because they are automated and require no additional data collection.Article SummaryIn this study, we found that two of the automated computer-aided risk scores are reasonably accurate for predicting the risk of COVID-19 and in-hospital mortality, respectively.They may be clinically useful as an early warning system at the time of admission especially to triage large numbers of unplanned hospital admissions because they are automated and require no additional data collection.Although we focused on in-hospital mortality (because we aimed to aid clinical decision making in the hospital), the impact of this selection bias needs to be assessed by capturing out-of-hospital mortality by linking death certification data and hospital data.We identified COVID-19 based on ICD-10 code ‘U071’ which was determined by COVID-19 swab test results (hospital or community) and clinical judgment and so our findings are constrained by the accuracy of these methods

show abstract

Section: Discussionsupporting

confidence: 88%

Accuracy of automated computer aided-risk scoring systems to estimate the risk of COVID-19 and in-hospital mortality: a retrospective cohort study

Faisal

Mohammed

Richardson

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…We have developed two automated risk equations to predict the patient’s risk of in-hospital mortality (CARM_N and CARM_NB) using NEWS only (CARM_N)6 and NEWS+blood test results (CARM_NB)7 following emergency medical admission to hospital. We found CARM_NB performed similar to consultant clinicians 8…”

Section: Introductionsupporting

confidence: 56%

“…We found CARM_NB performed similar to consultant clinicians. 8 NEWS2 was published in December 2017 as an update to NEWS 4 that considered new confusion or delirium and allocated three points (the maximum for a single variable). NEWS2 also offers two scales for oxygen saturation (scale 1 and scale 2).…”

Section: Strengths and Limitations Of This Studymentioning

confidence: 99%

Development and validation of automated computer-aided risk scores to predict in-hospital mortality for emergency medical admissions with COVID-19: a retrospective cohort development and validation study

et al. 2022

Self Cite

View full text Add to dashboard Cite

ObjectivesThere are no established mortality risk equations specifically for unplanned emergency medical admissions which include patients with SARS-19 (COVID-19). We aim to develop and validate a computer-aided risk score (CARMc19) for predicting mortality risk by combining COVID-19 status, the first electronically recorded blood test results and the National Early Warning Score (NEWS2).DesignLogistic regression model development and validation study.SettingTwo acute hospitals (York Hospital—model development data; Scarborough Hospital—external validation data).ParticipantsAdult (aged ≥16 years) medical admissions discharged over a 24-month period with electronic NEWS and blood test results recorded on admission. We used logistic regression modelling to predict the risk of in-hospital mortality using two models: (1) CARMc19_N: age+sex+NEWS2 including subcomponents+COVID19; (2) CARMc19_NB: CARMc19_N in conjunction with seven blood test results and acute kidney injury score. Model performance was evaluated according to discrimination (c-statistic), calibration (graphically) and clinical usefulness at NEWS2 thresholds of 4+, 5+, 6+.ResultsThe risk of in-hospital mortality following emergency medical admission was similar in development and validation datasets (8.4% vs 8.2%). The c-statistics for predicting mortality for CARMc19_NB is better than CARMc19_N in the validation dataset (CARMc19_NB=0.88 (95% CI 0.86 to 0.90) vs CARMc19_N=0.86 (95% CI 0.83 to 0.88)). Both models had good calibration (CARMc19_NB=1.01 (95% CI 0.88 to 1.14) and CARMc19_N:0.95 (95% CI 0.83 to 1.06)). At all NEWS2 thresholds (4+, 5+, 6+) model, CARMc19_NB had better sensitivity and similar specificity.ConclusionsWe have developed a validated CARMc19 scores with good performance characteristics for predicting the risk of in-hospital mortality. Since the CARMc19 scores place no additional data collection burden on clinicians, it may now be carefully introduced and evaluated in hospitals with sufficient informatics infrastructure.

show abstract

“…We have developed two automated risk equations to predict the patient’s risk of mortality (CARM_N & CARM_NB) using NEWS only (CARM_N) [6] and NEWS + blood test results (CARM_NB) [7] following emergency medical admission to hospital. We found CARM performed similar to consultant clinicians [8].…”

Section: Introductionsupporting

confidence: 55%

Development and validation of automated computer aided-risk score for predicting the risk of in-hospital mortality using first electronically recorded blood test results and vital signs for COVID-19 hospital admissions: a retrospective development and validation study

Faisal

Mohammed

Richardson

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Objectives: There are no established mortality risk equations specifically for unplanned emergency medical admissions which include patients with the novel coronavirus SARS-19 (COVID-19). We aim to develop and validate a computer-aided risk score (CARMc19) for predicting mortality risk by combining COVID-19 status, the first electronically recorded blood test results and latest version of the National Early Warning Score (NEWS2). Design: Logistic regression model development and validation study using a cohort of unplanned emergency medical admissions to hospital. Setting: York Hospital (YH) as model development dataset and Scarborough Hospital (SH) as model validation dataset. Participants: Unplanned adult medical admissions discharged over three months (11 March 2020 to 13 June 2020) from two hospitals (YH for model development; SH for external model validation) based on admission NEWS2 electronically recorded within 24 hours and or blood test results within 96 hours of admission. We used logistic regression modelling to predict the risk of in-hospital mortality using two models: 1) CARMc19_N: age + sex + NEWS2 including subcomponents + COVID19; 2) CARMc19_NB: CARMc19_N in conjunction with seven blood test results and acute kidney injury score. Model performance was evaluated according to discrimination (c-statistic), calibration (graphically), and clinical usefulness at NEWS2 thresholds of 4+, 5+, 6+. Results: The risk of in-hospital mortality following emergency medical admission was similar in development and validation datasets (8.4% vs 8.2%). The c-statistics for predicting mortality for Model CARMc19_NB is better than CARMc19_N in the validation dataset (CARMc19_NB = 0.88 (95%CI 0.86 to 0.90) vs CARMc19_N = 0.86 (95%CI 0.83 to 0.88)). Both models had good internal and external calibration (CARMc19_NB: 1.01 (95%CI 0.88 vs 1.14) & CARMc19_N: 0.95 (95%CI 0.83 to 1.06)). At all NEWS2 thresholds (4+, 5+, 6+) model CARMc19_NB had better sensitivity and similar specificity. Conclusions: We have developed a validated CARMc19 score with good performance characteristics for predicting the risk of in-hospital mortality following an emergency medical admission using the patients first, electronically recorded vital signs and blood tests results. Since the CARMc19 scores place no additional data collection burden on clinicians and is readily automated, it may now be carefully introduced and evaluated in hospitals with sufficient informatics infrastructure.

show abstract

A prospective study of consecutive emergency medical admissions to compare a novel automated computer-aided mortality risk score and clinical judgement of patient mortality risk

Cited by 5 publications

References 17 publications

Accuracy of automated computer aided-risk scoring systems to estimate the risk of COVID-19 and in-hospital mortality: a retrospective cohort study

Accuracy of automated computer aided-risk scoring systems to estimate the risk of COVID-19 and in-hospital mortality: a retrospective cohort study

Development and validation of automated computer-aided risk scores to predict in-hospital mortality for emergency medical admissions with COVID-19: a retrospective cohort development and validation study

Development and validation of automated computer aided-risk score for predicting the risk of in-hospital mortality using first electronically recorded blood test results and vital signs for COVID-19 hospital admissions: a retrospective development and validation study

Contact Info

Product

Resources

About