The optimal threshold for prompt clinical review: An external validation study of the national early warning score

Abstract

Aims and objectives: The aim of this study was to determine the optimal threshold for national early warning score in clinical practice. Background: The national early warning score is an aggregate early warning score aiming to predict patient mortality. Studies validating national early warning score did not use standardised patient outcomes or did not always include clinical workload in their results. Since all patients with a positive national early warning score require a clinical workup, it is crucial to determine the optimal threshold to limit false-positive alerts. Design: An external validation study using retrospectively collected data of patient admissions in six Belgian hospitals. Methods: We adhered to the STARD guideline for reporting. Two sample groups were selected: the cross-sectional sample (admitted patients, 1 day every 4 months) and the serious adverse event sample (all patients with unexpected death, cardiac arrest and unplanned admission to the intensive care unit). The maximum registered national early warning score value was collected in both groups. Predictive values were used as estimates for clinical workload. Results: We collected 1,523 in the cross-sectional sample and 390 patients in the serious adverse event sample. A national early warning score ≥5 had a positive predictive value of 6.8% and a negative predictive value of 99.5% to predict unexpected death, cardiac arrest with cardiopulmonary resuscitation or unplanned admission to intensive care (AUROC 0.841). The performance of national early warning score differed between outcome measures. Considering the predictive value, the optimal threshold for national early warning score is ≥5. Conclusions: We validated national early warning score to be applied in general hospital wards and confirmed the optimal threshold (≥5). Relevance to clinical practice: When a patient has a national early warning score <5, we may assume that in the next 24 hr this patient is less likely to die unexpectedly, receive cardiopulmonary resuscitation or be transferred to the ICU. Because of the significant number of false positives when national early warning score is ≥5, hospitals should create workable guidelines for clinical practice. | 4595 HAEGDORENS Et Al.

Early Warning Scores in Patients with Suspected COVID-19 Infection in Emergency Departments

“…The Coronavirus disease 2019 (COVID- 19), an infectious outbreak caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), has revolutionized healthcare systems around the world [1,2]. No country was entirely or even partially prepared to face the major consequences of the coronavirus infection, a highly contagious disease with a rapid rate of hospitalization.…”

“…The advantage of this type of score is the possibility of early detection of the risk of clinical deterioration, which has facilitated its use in multiple health systems, with a global implementation [16][17][18]. In addition, EWSs provide alert triggers that are adaptable to virtually any disease, making them highly versatile tools, capable of being used in different environments, physio-pathological situations, healthcare worker trainings or even latitudes [19][20][21][22].…”

A systematic review of the discrimination and absolute mortality predicted by the National Early Warning Scores according to different cut-off values and prediction windows

“…The discrimination of NEWS may be lower in COPD patients [43], and medication may also change NEWS performance; the AUC of NEWS for in-hospital mortality was lower in patients with suspected sepsis on hypertensive medication than those not on medication [44]. Despite these and other possible unconsidered confounders, we found only two studies that reported an AUC for 24-hour mortality <0.83, one used extensive imputation for missing data and was excluded [24], and the other selected patients by sampling instead of consecutive observations [23].…”

“…API = any paper that simply reported and assessment procedure, policy or implementation of an early warning score without outcomes data provided. sample of patients who did not die in hospital with those who died within 24-hours of NEWS measurement [23]. One study [24] that reported an AUC for 24-hour mortality of 0.65 was excluded from the final analysis because it used imputation so extensively for missing data.…”

Development and Validation of an Early Warning Model for Hospitalized COVID-19 Patients: A Multi-center Retrospective Cohort Study

“…In this setting, we consider a false positive rate (FPR) > 0.33 as clinically undesirable as it will lead to alert fatigue. 19 Moreover, other NEWS validation studies 20,21 have shown that the recommended triggers (i.e. NEWS = 5 or NEWS = 7 22 ) appear in the receiver operating characteristic curve for FPRs < 0.33.…”