Agreement and validity of electronic patient self-triage (eTriage) with nurse triage in two UK emergency departments: a retrospective study

Abstract

Background Triage and redirection of patients to alternative care providers is one tool used to overcome the growing issue of crowding in emergency departments (EDs). Electronic patient self-triage (eTriage) may reduce waiting times and required face-to-face contact. There are limited studies into its efficacy, accuracy and validity in an ED setting.Objectives The aim of this study was to assess the agreement and validity of eTriage with a reference standard of nurse face-to-face triage. A secondary aim was to assess the ability of both systems to predict high and low acuity outcomes.Design This was a retrospective study conducted over 8 months in two UK hospitals. Inclusion criteria were all ambulatory patients aged ≥18. All patients completed an eTriage and nurse-led triage using the Manchester Triage System (MTS). Main ResultsDuring the study period, 43 788 adult patients attended one of the two ED sites and 26 757 used eTriage. A total of 1424 patient episodes had no recorded MTS and were excluded from the study leaving 25 333 paired triages for the final cohort. Agreement between eTriage and nurse triage was low with a weighted Kappa coefficient of 0.14 (95% CI, 0.14-0.15) with an associated weak positive correlation (r s 0.321). Level of undertriage by eTriage compared with nurse triage was 10.1%, and overtriage was 59.2%. The sensitivity for prediction of high acuity outcomes was 88.5% (95% CI, 77.9-95.3%) for eTriage and 53.8% (95% CI 41.1-66.0%) for nurse MTS. The specificity for predicting low risk patients was 88.5% (95% CI, 87.4-89.5%) for eTriage and 80.6% (95% CI, 79.3-81.8%) for nurse MTS. ConclusionAgreement and correlation of eTriage with the reference standard of nurse MTS was low; patients using eTriage tended to over triage when compared to the triage nurse. eTriage had a higher sensitivity for high acuity presentations and demonstrated similar specificity for low acuity presentations when compared to triage nurse MTS. Further work is necessary to validate eTriage as a potential tool for safe redirection of ED attenders to alternative care providers.

A scoping review on the use and usefulness of online symptom checkers and triage systems: How to proceed?

“…Research examining safety and accuracy of these systems highlights the persistence of a risk-averse disposition (12,(15)(16)(17). An effort is being made to evolve to more balanced algorithms, but some studies report that this evolution currently comes at the expense of the technology's safety, which should always remain their priority (19)(20)(21).…”

“…Digital triage services are generally considered risk averse (15)(16)(17), favoring sensitivity over specificity, often over-triaging, assigning a higher urgency level than is required. A systematic review by Chambers et al (12) concluded that the available studies reported no evidence of a detrimental effect on patient safety in both simulated and real settings.…”

Leveraging Large-Scale Electronic Health Records and Interpretable Machine Learning for Clinical Decision Making at the Emergency Department: Protocol for System Development and Validation

“…Candidate variables have been identified based on expert opinions as well as relevant literature [18,30,[34][35][36]. Moreover, we have sought input from clinicians and informaticians familiar with the raw data to determine which features are feasible to extract and construct from the sources.…”

Triage Errors in Primary and Pre–Primary Care

“…When applied in emergency department settings, symptom checkers were reported to be inadequately sensitive to emergency cases, with triage accuracy between 45%-75% of total patients [ 27 - 30 ]. However, in a recent study using digital patient self-triage in a hospital emergency department, a digital tool showed higher sensitivity to high-acuity conditions and similar specificity for low-acuity conditions when compared with standard nurse triage using the Manchester Triage System; it also tended to result in overtriage of patients when compared with standard nurse triage [ 31 ].…”