Artificial Intelligence-Based Application Provides Accurate Medical Triage Advice When Compared to Consensus Decisions of Healthcare Providers

Delshad, Sean; Dontaraju, Venkata Satish; Chengat, Vipindas

doi:10.7759/cureus.16956

Cited by 19 publications

(26 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The ODISSEE platform is based on decisional flowcharts with binary answers leading to specific advised destinations. Currently, several other different models of self-triage tools exist based on more sophisticated models driven by artificial intelligence (AI) [ 11 , 34 ]. As AI-driven tools seem to present promising results in terms of safety, we should question the interest in developing classical algorithmic apps versus AI-driven apps.…”

Section: Discussionmentioning

confidence: 99%

Patients’ self-triage for unscheduled urgent care: a preliminary study on the accuracy and factors affecting the performance of a Belgian self-triage platform

Gilbert

Diep

Boufraioua

et al. 2022

BMC Health Serv Res

View full text Add to dashboard Cite

Background Management of unscheduled urgent care is a complex concern for many healthcare providers. Facing the challenge of appropriately dispatching unscheduled care, primary and emergency physicians have collaboratively implemented innovative strategies such as telephone triage. Currently, new original solutions tend to emerge with the development of new technologies. We created an interactive patient self-triage platform, ODISSEE, and aimed to explore its accuracy and potential factors affecting its performance using clinical case scenarios. Methods The ODISSEE platform was developed based on previously validated triage protocols for out-of-hours primary care. ODISSEE is composed of 18 icons leading to algorithmic questions that finally provide an advised orientation (emergency or primary care services). To investigate ODISSEE performance, we used 100 clinical case scenarios, each associated with a preestablished orientation determined by a group of experts. Fifteen volunteers were asked to self-triage with 50 randomly selected scenarios using ODISSEE on a digital tablet. Their triage results were compared with the experts’ references. Results The 15 participants performed a total of 750 self-triages, which matched the experts references regarding the level of care in 85.6% of the cases. The orientation was incorrect in 14.4%, with an undertriage rate of 1.9% and an overtriage rate of 12.5%. The tool’s specificity and sensitivity to advise participants on the appropriate level of care were 69% (95% CI: 64—74) and 97% (95% CI: 95—98) respectively. When combined with advice on the level of urgency, the tool only found the correct orientation in 68.4% with 9.2% of undertriages and 22.4% of overtriages. Some participant characteristics and the types of medical conditions demonstrated a significant association with the tool performance. Conclusion Self-triage apps, such as the ODISSEE platform, could represent an innovative method to allow patients to self-triage to the most appropriate level of care. This study based on clinical vignettes highlights some positive arguments regarding ODISSEE safety, but further research is needed to assess the generalizability of such tools to the population without equity issues.

show abstract

Section: Discussionmentioning

confidence: 99%

Patients’ self-triage for unscheduled urgent care: a preliminary study on the accuracy and factors affecting the performance of a Belgian self-triage platform

Gilbert

Diep

Boufraioua

et al. 2022

BMC Health Serv Res

View full text Add to dashboard Cite

show abstract

“…Table 1 shows the main characteristics of included studies published between 2014 and 2022. Six [11][12][13][14][15][16] were conducted by researchers based in the USA, three in the UK [17][18][19], two in Australia [20,21], two in Canada [22,23], one in the Netherlands [24] and one in Hong Kong [25]. Eight studies [11,12,[14][15][16][19][20][21] used standardised patient vignettes; several were inspired by or included the 45 vignettes used by Semigran et al [11,12] The remaining seven studies used data from real patients either through their medical health records [13,25] or direct input by users [17,18,[22][23][24] in different settings, including primary care and emergent care settings.…”

Section: Characteristics Of Included Studiesmentioning

confidence: 99%

“…Seven studies evaluated a single OSC [14,[16][17][18][22][23][24], while the other eight tested and compared the performance of two [25] to 36 [20] OSCs. Where provided, the most common justifications for selection were language (English), level of popularity among users, and accessibility (free).…”

Section: Characteristics Of Included Studiesmentioning

confidence: 99%

“…Semigran et al [11] -Semigran et al [12] -Berry et al [13] -Gilbert et al [14] -Hill et al [20] -Ceney et al [19] -Gilbert et al [21] Overall diagnostic accuracy was deemed to be low, and always lower than the comparator Strong Triage accuracy -Poote et al [17] ±Semigran et al [11] -Verzantvoort et al [24] -Berry et al [13] ±Gilbert et al [14] -Hill et al [20] -Yu et al [25]-Ceney et al[19] +Chan et al[23] +Delshad et al[16] -Gilbert et al[21] ±Schmieding et al[15] Inconsistent findings, including within studies. Great variations between OSCs.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Triage and Diagnostic Accuracy of Online Symptom Checkers: Systematic Review (Preprint)

Sasco¹,

El-Osta²,

Alaa³

et al. 2022

Preprint

View full text Add to dashboard Cite

BACKGROUND In the context of a deepening global shortage of health workers, and particularly the COVID-19 pandemic, there is growing international interest in and use of online symptom checkers (OSCs). However, the evidence surrounding the safety and accuracy of OSCs remains inconclusive so far. The triage and diagnostic accuracy of these tools is an essential aspect that needs to be addressed before pushing any further implementation. OBJECTIVE This systematic review aimed to summarize the existing peer-reviewed literature evaluating the triage accuracy (directing users to appropriate services based on their presenting symptoms) and diagnostic accuracy of OSCs aimed at lay users for general health concerns. METHODS Searches were conducted in Medline, Embase, CINAHL, HMIC and Web of Science. We included peer-reviewed studies published in English between 1 January 2010 and 17 February 2022 with a quantitative assessment of triage and/or diagnostic accuracy of OSCs directed at lay users. We excluded tools supporting health professionals, and disease- or speciality-specific OSCs. Screening and data extraction were carried out independently by two reviewers for each study. We performed a descriptive narrative synthesis. RESULTS 21,284 studies were screened and 15 were included. Six studies reported on both triage and diagnostic accuracy, eight focused on triage accuracy, and one on diagnostic accuracy. Diagnostic and triage accuracy varied between studies and OSCs; most studies showed suboptimal diagnostic and triage accuracy. Frequency and urgency of the condition were the main variables that affected the levels of diagnostic and triage accuracy, along with specific features of the OSCs. The impact of each variable differed across tools and studies, making it difficult to draw any solid conclusions. Included studies had either a moderate or high risk of bias according to the revised tool for the Quality Assessment of Diagnostic Accuracy Studies 2. CONCLUSIONS While OSCs have significant potential to provide accessible and accurate health advice and triage recommendations to users, more research is needed to validate their triage and diagnostic accuracy prior to wide scale adoption in community and healthcare settings. Future studies should aim to use a common methodology and/or agreed standard for evaluation to facilitate objective benchmarking and validation.

show abstract

“…Also, further collaboration with legal experts should be considered if a system is to be deployed. While HITL systems still require human supervision and the efficiency increment varies from case to case, HITL systems have been shown to improve both accuracy and efficiency in radiology reporting [ 49 ] and general medical triaging [ 8 ]. The degree of human involvement can also be varied according to the confidence of the network, further improving efficiency.…”

Section: Limitations and Challengesmentioning

confidence: 99%

A Proposed Framework for Machine Learning-Aided Triage in Public Specialty Ophthalmology Clinics in Hong Kong

Vardhanabhuti

Tsougenis

et al. 2021

Ophthalmol Ther

View full text Add to dashboard Cite

Artificial Intelligence-Based Application Provides Accurate Medical Triage Advice When Compared to Consensus Decisions of Healthcare Providers

Cited by 19 publications

References 13 publications

Patients’ self-triage for unscheduled urgent care: a preliminary study on the accuracy and factors affecting the performance of a Belgian self-triage platform

Patients’ self-triage for unscheduled urgent care: a preliminary study on the accuracy and factors affecting the performance of a Belgian self-triage platform

Triage and Diagnostic Accuracy of Online Symptom Checkers: Systematic Review (Preprint)

A Proposed Framework for Machine Learning-Aided Triage in Public Specialty Ophthalmology Clinics in Hong Kong

Contact Info

Product

Resources

About