Triage accuracy of online symptom checkers for Accident and Emergency Department patients

Yu, Stephanie; Ma, Andre; Tsang, Vivian Hiu Man; Chung, Lulu Suet Wing; Leung, Sammual Yu‐Lut; Leung, Ling‐Pong

doi:10.1177/1024907919842486

Cited by 38 publications

(55 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Study population size ranged from 27 to 214 patients or vignettes. [17,21] Three studies were conducted in USA, [1,22,23] and one each in Australia, [19] Canada, [18] Hong Kong, [20] Spain,[17] and the United Kingdom. [16] The remaining two were multinational studies.…”

Section: Characteristics Of Participants and Interventionsmentioning

confidence: 99%

The diagnostic and triage accuracy of digital and online symptom checker tools: a systematic review

Wallace

Chan

Chidambaram

et al. 2021

Preprint

View full text Add to dashboard Cite

Objective To evaluate the accuracy of digital and online symptom checkers in providing diagnoses and appropriate triage advice. Design Systematic review. Data sources Medline and Web of Science were searched up to 15 February 2021. Eligibility criteria for study selection Prospective and retrospective cohort, vignette, or audit studies that utilised an online or application-based service designed to input symptoms and biodata in order to generate diagnoses, health advice and direct patients to appropriate services were included. Main outcome measures The primary outcomes were (1) the accuracy of symptom checkers for providing the correct diagnosis and (2) the accuracy of subsequent triage advice given. Data extraction and synthesis Data extraction and quality assessment (using the QUADAS-2 tool) were performed by two independent reviewers. Owing to heterogeneity of the studies, meta-analysis was not possible. A narrative synthesis of the included studies and pre-specified outcomes was completed. Results Of the 177 studies retrieved, nine cohort studies and one cross-sectional study met the inclusion criteria. Symptom checkers evaluated a variety of medical conditions including ophthalmological conditions, inflammatory arthritides and HIV. 50% of the studies recruited real patients, while the remainder used simulated cases. The diagnostic accuracy of the primary diagnosis was low (range: 19% to 36%) and varied between individual symptom checkers, despite consistent symptom data input. Triage accuracy (range: 48.8% to 90.1%) was typically higher than diagnostic accuracy. Of note, one study found that 78.6% of emergency ophthalmic cases were under-triaged. Conclusions The diagnostic and triage accuracy of symptom checkers are variable and of low accuracy. Given the increasing push towards population-wide digital health technology adoption, reliance upon symptom checkers in lieu of traditional assessment models, poses the potential for clinical risk. Further primary studies, utilising improved study reporting, core outcome sets and subgroup analyses, are warranted to demonstrate equitable and non-inferior performance of these technologies to that of current best practice. PROSPERO registration number CRD42021271022.

show abstract

Section: Characteristics Of Participants and Interventionsmentioning

confidence: 99%

The diagnostic and triage accuracy of digital and online symptom checker tools: a systematic review

Wallace

Chan

Chidambaram

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The online platform is a beneficial resource for those experiencing mental health issues and comprises of numerous provisions for self-regulation of one's personal mental health (McCosker, 2018). Online platforms offer accessibility to symptom checkers (Yu et al, 2019), counselling (March et al, 2018;Schlenk, 2018), peer-to-peer support (Prescott et al, , 2018Hanley et al, 2019) social media support groups (Radovic et al, 2019) and chat rooms (Kiyimba et al, 2019), amongst many others. Because of increased usage of health-care settings and the ease of access which online platforms facilitate, clinicians and academics alike extensively explore the efficacy of e-health as a mental health intervention .…”

Section: Online Help Seekingmentioning

confidence: 99%

Online mental health communities, self-efficacy and transition to further support

Prescott

Rathbone

Hanley

2020

MHRJ

View full text Add to dashboard Cite

Purpose This study is exploratory research which aims to understand how users gain support from the online mental health community (OMHC) 18 percent and whether engagement with this community may possibly lead to increased self-efficacy. Design/methodology/approach In total, 128 users of an OMHC, 18 percent, completed an online questionnaire that asked open-ended questions about the community and how users engaged with it. The results were analysed using quantitative and qualitative methods. Findings Based on the construct of self-efficacy within social cognitive theory, it is evident that the platform provides users with increased self-efficacy and encourages further support seeking in a professional capacity, either via an online or offline platform. Originality/value OMHCs provide a therapeutic, peer-to-peer space for users in times of crisis which have the possibility to increase self-efficacy when engaged with. However, users must acknowledge that although the online platform is an efficacious resource, it cannot be used as a principal proxy for offline treatment.

show abstract

“…For this to happen, healthcare professionals and the wider public must have confidence in the performance of symptom checkers and applications of AI to medicine more generally. Previous work has investigated the diagnostic and triage accuracy of symptom checkers ( Semigran et al, 2015 ; Millenson et al, 2018 ; Chambers et al, 2019 ; Yu et al, 2019 ). In particular, ( Semigran et al, 2015 ) assessed 23 commercial symptom checker apps and highlighted significant variation in terms of clinical accuracy.…”

Section: Introductionmentioning

confidence: 99%

A Comparison of Artificial Intelligence and Human Doctors for the Purpose of Triage and Diagnosis

Baker

Perov

Middleton

et al. 2020

Front. Artif. Intell.

View full text Add to dashboard Cite

AI virtual assistants have significant potential to alleviate the pressure on overly burdened healthcare systems by enabling patients to self-assess their symptoms and to seek further care when appropriate. For these systems to make a meaningful contribution to healthcare globally, they must be trusted by patients and healthcare professionals alike, and service the needs of patients in diverse regions and segments of the population. We developed an AI virtual assistant which provides patients with triage and diagnostic information. Crucially, the system is based on a generative model, which allows for relatively straightforward re-parameterization to reflect local disease and risk factor burden in diverse regions and population segments. This is an appealing property, particularly when considering the potential of AI systems to improve the provision of healthcare on a global scale in many regions and for both developing and developed countries. We performed a prospective validation study of the accuracy and safety of the AI system and human doctors. Importantly, we assessed the accuracy and safety of both the AI and human doctors independently against identical clinical cases and, unlike previous studies, also accounted for the information gathering process of both agents. Overall, we found that the AI system is able to provide patients with triage and diagnostic information with a level of clinical accuracy and safety comparable to that of human doctors. Through this approach and study, we hope to start building trust in AI-powered systems by directly comparing their performance to human doctors, who do not always agree with each other on the cause of patients’ symptoms or the most appropriate triage recommendation.

show abstract

Triage accuracy of online symptom checkers for Accident and Emergency Department patients

Cited by 38 publications

References 15 publications

The diagnostic and triage accuracy of digital and online symptom checker tools: a systematic review

The diagnostic and triage accuracy of digital and online symptom checker tools: a systematic review

Online mental health communities, self-efficacy and transition to further support

A Comparison of Artificial Intelligence and Human Doctors for the Purpose of Triage and Diagnosis

Contact Info

Product

Resources

About