Prognostic accuracy of triage tools for adults with suspected COVID-19 in a pre-hospital setting: an observational cohort study

Abstract

Study Objective Tools proposed to triage patient acuity in COVID-19 infection have only been validated in hospital populations. We estimated the accuracy of five risk-stratification tools recommended to predict severe illness and compare accuracy to existing clinical decision-making in a pre-hospital setting. Methods An observational cohort study using linked ambulance service data for patients attended by EMS crews in the Yorkshire and Humber region of England between 18th March 2020 and 29th June 2020 was conducted to assess performance of the PRIEST tool, NEWS2, the WHO algorithm, CRB-65 and PMEWS in patients with suspected COVID-19 infection. The primary outcome was death or need for organ support. Results Of 7549 patients in our cohort, 17.6% (95% CI:16.8% to 18.5%) experienced the primary outcome. The NEWS2, PMEWS, PRIEST tool and WHO algorithm identified patients at risk of adverse outcomes with a high sensitivity (>0.95) and specificity ranging from 0.3 (NEWS2) to 0.41 (PRIEST tool). The high sensitivity of NEWS2 and PMEWS was achieved by using lower thresholds than previously recommended. On index assessment, 65% of patients were transported to hospital and EMS decision to transfer patients achieved a sensitivity of 0.84 (95% CI 0.83 to 0.85) and specificity of 0.39 (95% CI 0.39 to 0.40). # Conclusion Use of NEWS2, PMEWS, PRIEST tool and WHO algorithm could improve sensitivity of EMS triage of patients with suspected COVID-19 infection. Use of the PRIEST tool would improve sensitivity of triage without increasing the number of patients conveyed to hospital.

Prognostic accuracy of triage tools for adults with suspected COVID-19 in a middle-income setting: an observational cohort study

“…In a UK prehospital study, the PRIEST score achieved a c-statistic of 0.83 (0.82 to 0.84), sensitivity of 0.97 (95%CI: 0.97 to 0.97) and specificity of 0.41 (95%CI: 0.40 to 0.41). 29 In this study cohort, the PRIEST score (without inclusion of performance status) achieved the best overall discrimination (c-statistic 0.82 (95% CI 0.82 to 0.82)) and sensitivity 0.83 (95% CI 0.83 to 0.83) and specificity of 0.65 (95% CI 0.65 to 0.66) at the recommended threshold to guide hospital admission. Although the discrimination is similar to previous validation studies the differences in accuracy are not explainable solely by the lower prevalence in this study cohort.…”

“…The lower risk of death and ICU admission in this study may reflect the role telephone and other pre-hospital triage had in reducing ED attendances of lower-risk patients in the UK and other settings and the lower severity associated with the Omicron wave. 29,33 This highlights the need to validate triage tools in different settings and waves to ensure that provide accurate predictions of risk. 3 Current clinical decision-making to admit patients from the ED in this study cohort achieved a sensitivity of 0.77 (95% CI 0.76 to 0.78) and specificity 0.88 (95% CI 0.87 to 0.88) for the primary outcome.…”

“…34 The score has been externally validated in a UK pre-hospital alpha wave cohort and a cohort of 306 patients presenting to EDs in the USA in the winter of 2020-21. 29,35 In the development study, the PRIEST score achieved a c-statistic of 0.80 (95%CI 0.79 to 0.81) and, at the recommended threshold of a score above four points, a sensitivity 0.98 (95% 0.97 to 0.98) and specificity 0.34 (95% 0.34 to 0.35) for a composite outcome of death or organ support. 7 In the American validation study, the score achieved a c-statistic of 0.86 (95% CI: 0.81 to 0.91) and sensitivity of 97.7% (95%CI: 93.2% to 100%) and specificity of 47.2% (95% CI: 41.1% to 53.2%) for a similar outcome 35 .…”

“…We a priori assessed the estimated precision of the area under the ROC curve based on a likely 5% event rate for a minimum estimated cohort of 6000 patients. Assuming an AUC of 0.8, based on previous triage tool validation studies, 29 this sample size would provide an acceptable 95% confidence interval width of 0.06 (95% CI 0.77 to 0.83). In the event the sample size was well in excess of this, and the reported confidence interval width indicates the range in which the population parameter is likely to reside.…”

“…As with previous studies, triage tools performed better in this study when predicting death compared to the components of the primary outcome indicating need for organ support (ICU admission in this study). 7,8,29 Strengths and limitations This is the first study to use a large cohort of patients identified using routinely collected electronic health care data to validate triage tools in patients with suspected COVID in the Western Cape in the Emergency Department setting. The use of a cohort of patients with suspected infection is important, as this reflects the population which ED staff must clinically triage.…”