Evaluation of symptom checkers for self diagnosis and triage: audit study

Abstract

ObjeCtiveTo determine the diagnostic and triage accuracy of online symptom checkers (tools that use computer algorithms to help patients with self diagnosis or self triage). DesignAudit study.setting Publicly available, free symptom checkers.PartiCiPants 23 symptom checkers that were in English and provided advice across a range of conditions. 45 standardized patient vignettes were compiled and equally divided into three categories of triage urgency: emergent care required (for example, pulmonary embolism), non-emergent care reasonable (for example, otitis media), and self care reasonable (for example, viral upper respiratory tract infection). Main OutCOMe MeasuresFor symptom checkers that provided a diagnosis, our main outcomes were whether the symptom checker listed the correct diagnosis first or within the first 20 potential diagnoses (n=770 standardized patient evaluations). For symptom checkers that provided a triage recommendation, our main outcomes were whether the symptom checker correctly recommended emergent care, non-emergent care, or self care (n=532 standardized patient evaluations). resultsThe 23 symptom checkers provided the correct diagnosis first in 34% (95% confidence interval 31% to 37%) of standardized patient evaluations, listed the correct diagnosis within the top 20 diagnoses given in 58% (55% to 62%) of standardized patient evaluations, and provided the appropriate triage advice in 57% (52% to 61%) of standardized patient evaluations. Triage performance varied by urgency of condition, with appropriate triage advice provided in 80% (95% confidence interval 75% to 86%) of emergent cases, 55% (47% to 63%) of non-emergent cases, and 33% (26% to 40%) of self care cases (P<0.001). Performance on appropriate triage advice across the 23 individual symptom checkers ranged from 33% (95% confidence interval 19% to 48%) to 78% (64% to 91%) of standardized patient evaluations.COnClusiOns Symptom checkers had deficits in both triage and diagnosis. Triage advice from symptom checkers is generally risk averse, encouraging users to seek care for conditions where self care is reasonable.

The patient’s dilemma: attending the emergency department with a minor illness

“…A virtual visit to "Dr Google" is likely to encourage, rather than decrease, non-urgent visits. 4 Anxiety, the need for reassurance, or believing their problem needs the resources of a hospital are common findings in studies of patients with non-urgent complaints visiting emergency departments. 1 5 In the UK study, nearly half of such patients said they needed to see a doctor "as soon as possible.…”

Availability of prescription drugs for bipolar disorder at online pharmacies

“…Symptom checklists are a mainstay of DTC campaigns, and not only promote self-diagnosis but urge consumers to demand a specific drug from their providers (Ebeling, 2011;Donohue, 2006). Additionally, the availability and use of medical apps and online symptom checkers for self-diagnosis is increasing rapidly (Semigran et al, 2015;Lupton and Jutel, 2015) with 1/3 of US adults using resources on the Internet to self-diagnose (Kuehn, 2013). A vision of great concern depicts people alternating roles between consumers when they purchase prescription drugs without a prescription, and patients when they turn to the traditional medical system to treat numerous potential complications of self-prescribing (Lineberry and Bostwick, 2004).…”

Evaluating the impact of the Eastern Academic Health Science Network Transformation Support Unit

“…The evidence they report is inconclusive in terms of the impact on managing demand on professional time, patient experience and health outcomes. Diagnosis apps are found to be not always accurate and interactive symptom checkers may recommend professional care unnecessarily (Semigran et al, 2015, Bierbrier et al, 2014. However, online information appeared to help patients to manage their conditions and have more productive conversations with their GPs.…”