Validation of pneumonia prognostic scores in a statewide cohort of hospitalised patients with COVID‐19

Abstract

Objective We aimed to externally validate the predictive performance of two recently developed COVID‐19‐specific prognostic tools, the COVID‐GRAM and CALL scores, and prior prognostic scores for community‐acquired pneumonia (CURB‐65), viral pneumonia (MuBLSTA) and H1N1 influenza pneumonia (Influenza risk score) in a contemporary US cohort. Methods We included 257 hospitalised patients with laboratory‐confirmed COVID‐19 pneumonia from three teaching hospitals in Rhode Island. We extracted data from within the first 24 hours of admission. Variables were excluded if values were missing in >20% of cases, otherwise, missing values were imputed. One hundred and fifteen patients with complete data after imputation were used for the primary analysis. Sensitivity analysis was performed after the exclusion of one variable (LDH) in the complete dataset (n = 257). Primary and secondary outcomes were in‐hospital mortality and critical illness (mechanical ventilation or death), respectively. Results Only the areas under the receiver‐operating characteristic curves (RO‐AUC) of COVID‐GRAM (RO‐AUC = 0.775, 95% CI 0.525‐0.915) for in‐hospital death, and CURB65 for in‐hospital death (RO‐AUC = 0.842, 95% CI 0.674‐0.932) or critical illness (RO‐AUC = 0.766, 95% CI 0.584‐0.884) were significantly better than random. Sensitivity analysis yielded similar trends. Calibration plots showed better agreement between the estimated and observed probability of in‐hospital death for CURB65, compared with COVID‐GRAM. The negative predictive value (NPV) of CURB65 ≥2 was 97.2% for in‐hospital death and 88.1% for critical illness. Conclusions The COVID‐GRAM score demonstrated acceptable predictive performance for in‐hospital death. The CURB65 score had better prognostic utility for in‐hospital death and critical illness. The high NPV of CURB65 values ≥2 may be useful in triaging and allocation of resources.

Predictors of severity and mortality among patients hospitalized with COVID-19 in Rhode Island

“…To ensure that we had a representative sample, we compared the weekly case fatality rates between the study sample and all COVID-19 patients admitted to the hospitals during the study period. We did not observe a statistical significance (Wilcoxon test P>0.1) [10].…”

“…Sixty (23%) participants were admitted from a nursing facility; 52/259 (20%) had additional emergency room (ER) visits in the 15 days before hospitalization. The median length of stay for all patients was 8 days [IQR, [5][6][7][8][9][10][11][12][13][14][15]. ICU admission was required in 74/259 (28%); 42/259 (16%) patients required mechanical ventilation with a median time of 7 days [IQR, [3][4][5][6][7][8][9][10][11][12] on a ventilator.…”

Clinical Management of Adult Patients with COVID-19 Outside Intensive Care Units: Guidelines from the Italian Society of Anti-Infective Therapy (SITA) and the Italian Society of Pulmonology (SIP)

“…The National Early Warning Score-2 (NEWS-2) based on six physiological parameters was also evaluated for its prognostic performance in hospitalized patients with COVID-19, but any possible impact on hospitalization decisions remains to be assessed [33,37,38]. Besides the aforementioned, also many other prognostic scores/models have been proposed that, for the purpose of the present question, suffer from the same limitation, i.e., it is not possible to directly extrapolate their performance to hospitalization decisions [39][40][41][42][43][44][45][46][47][48][49][50][51][52][53].…”

Evaluating risk stratification scoring systems to predict mortality in patients with COVID-19

“…Within all AI modelling papers, Meng et al 33 and Vaid et al 10 were the only studies that conducted external validation. External validation is an important step to verify the effectiveness of the model in patient population.…”