A study of automated self-assessment in a primary care student health centre setting

Poote, Aimee; French, David P.; Dale, Jeremy; Powell, John

doi:10.1177/1357633x14529246

Cited by 31 publications

(52 citation statements)

References 11 publications

(19 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Semigran and colleagues recently tested 23 free online SCs with standardised patient vignettes; they found that correct diagnoses headed the result list in 34% of evaluations, and appropriate triage advice was provided in 57% of cases . In a study in a British primary care clinic, a self‐assessment tool recommended a more urgent level of care than the general practitioner in 56% of cases and less urgent advice for only 5% of consultations …”

mentioning

confidence: 99%

The quality of diagnosis and triage advice provided by free online symptom checkers and apps in Australia

Hill¹,

Sim²,

Mills³

2020

Medical Journal of Australia

214

View full text Add to dashboard Cite

Objectives To investigate the quality of diagnostic and triage advice provided by free website and mobile application symptom checkers (SCs) accessible in Australia. Design 36 SCs providing medical diagnosis or triage advice were tested with 48 medical condition vignettes (1170 diagnosis vignette tests, 688 triage vignette tests). Main outcome measures Correct diagnosis advice (provided in first, the top three or top ten diagnosis results); correct triage advice (appropriate triage category recommended). Results The 27 diagnostic SCs listed the correct diagnosis first in 421 of 1170 SC vignette tests (36%; 95% CI, 31–42%), among the top three results in 606 tests (52%; 95% CI, 47–59%), and among the top ten results in 681 tests (58%; 95% CI, 53–65%). SCs using artificial intelligence algorithms listed the correct diagnosis first in 46% of tests (95% CI, 40–57%), compared with 32% (95% CI, 26–38%) for other SCs. The mean rate of first correct results for individual SCs ranged between 12% and 61%. The 19 triage SCs provided correct advice for 338 of 688 vignette tests (49%; 95% CI, 44–54%). Appropriate triage advice was more frequent for emergency care (63%; 95% CI, 52–71%) and urgent care vignette tests (56%; 95% CI, 52–75%) than for non‐urgent care (30%; 95% CI, 11–39%) and self‐care tests (40%; 95% CI, 26–49%). Conclusion The quality of diagnostic advice varied between SCs, and triage advice was generally risk‐averse, often recommending more urgent care than appropriate.

show abstract

mentioning

confidence: 99%

The quality of diagnosis and triage advice provided by free online symptom checkers and apps in Australia

Hill¹,

Sim²,

Mills³

2020

Medical Journal of Australia

214

View full text Add to dashboard Cite

show abstract

“…The remaining studies in this group evaluated symptom checkers that covered a wider range of common urgent-care symptoms. Studies evaluated either a single system 21,22,24,27 or multiple systems. 6,13 We found only one study of a symptom checker specifically intended for the assessment of children's symptoms, a development of the Strategy for Off-Site Rapid Triage (SORT) system for influenza-like illnesses.…”

Section: Characteristics Of Included Studiesmentioning

confidence: 99%

“…Limitations of the studies included not being based on real patient data, 27 covering only a limited range of conditions, 16,23 and sampling a young and healthy population (students) that was not representative of the general population of users of the urgent-care system. 22 Studies of e-consultation systems did not generally collect data on those respondents who decided not to seek an appointment, which limited their ability to assess any impact on safety for this group. Overall, the evidence should be interpreted cautiously as indicating no evidence of a detrimental impact on safety rather than evidence of no detrimental effect.…”

Section: Safetymentioning

confidence: 99%

“…The ability to distinguish conditions of different degrees of urgency and advise patients on an appropriate course of action is a vital component of symptom checkers and e-consultation services and is closely linked to both safety and service use. Six included studies 6,7,21,22,26,27 reported on this outcome, and one 30 evaluated a 'general purpose' symptom checker ( Table 10). As with diagnostic accuracy, diverse methodologies and outcome measures were used.…”

Section: Accuracy Of Disposition (Triage and Signposting To Appropriamentioning

confidence: 99%

See 1 more Smart Citation

Digital and online symptom checkers and assessment services for urgent care to inform a new digital platform: a systematic review

Chambers

Cantrell

Johnson

et al. 2019

Health Serv Deliv Res

View full text Add to dashboard Cite

Background Digital and online symptom checkers and assessment services are used by patients seeking guidance about health problems. NHS England is planning to introduce a digital platform (NHS111 Online) to operate alongside the NHS111 urgent-care telephone service. This review focuses on digital and online symptom checkers for urgent health problems. Objectives This systematic review was commissioned to provide NHS England with an independent review of previous research in this area to inform strategic decision-making and service design. Data sources Focused searches of seven bibliographic databases were performed and supplemented by phrase searching for names of symptom checker systems and citation searches of key included studies. The bibliographic databases searched were MEDLINE, EMBASE, The Cochrane Library, CINAHL (Cumulative Index to Nursing and Allied Health Literature), HMIC (Health Management Information Consortium), Web of Science and the Association of Computing Machinery (ACM) Digital Library, from inception up to April 2018. Review methods Brief inclusion criteria were (1) population – general population seeking information online or digitally to address an urgent health problem; (2) intervention – any online or digital service designed to assess symptoms, provide health advice and direct patients to appropriate services; and (3) comparator – telephone or face-to-face assessment, comparative performance in tests or simulations (studies with no comparator were included if they reported relevant outcomes). Outcomes of interest included safety, clinical effectiveness, costs or cost-effectiveness, diagnostic and triage accuracy, use of and contacts with health services, compliance with advice received, patient/carer satisfaction, and equity and inclusion. Inclusion was not restricted by study design. Screening studies for inclusion, data extraction and quality assessment were carried out by one reviewer with a sample checked for accuracy and consistency. Final decisions on study inclusion were taken by consensus of the review team. A narrative synthesis of the included studies was performed and structured around the predefined research questions and key outcomes. The overall strength of evidence for each outcome was classified as ‘stronger’, ‘weaker’, ‘conflicting’ or ‘insufficient’, based on study numbers and design. Results In total, 29 publications describing 27 studies were included. Studies were diverse in their design and methodology. The overall strength of the evidence was weak because it was largely based on observational studies and with a substantial component of non-peer-reviewed grey literature. There was little evidence to suggest that symptom checkers are unsafe, but studies evaluating their safety were generally short term and small scale. Diagnostic accuracy was highly variable between different systems but was generally low. Algorithm-based triage tended to be more risk averse than that of health professionals. Inconsistent evidence was found on effects on service use. There was very limited evidence on patients’ reactions to online triage advice. The studies showed that younger and more highly educated people are more likely to use these services. Study participants generally expressed high levels of satisfaction with digital and online triage services, albeit in uncontrolled studies. Limitations Findings from symptom checker systems for specific conditions may not be applicable to more general systems and vice versa. Studies of symptom checkers as part of electronic consultation systems in general practice were also included, which is a slightly different setting from a general ‘digital 111’ service. Most studies were screened by one reviewer. Conclusions Major uncertainties surround the probable impact of digital 111 services on most outcomes. It will be important to monitor and evaluate the services using all available data sources and by commissioning high-quality research. Future work Priorities for research include comparisons of different systems, rigorous economic evaluations and investigations of patient pathways. Study registration The study is registered as PROSPERO CRD42018093564. Funding The National Institute for Health Research Health Services and Delivery Research programme.

show abstract

“…In [1] there is described a self-assessment system prior to a face-to-face consultation with the general practitioner (GP). In this study a perfect agreement regarding the advice provided, between the GP and the system was found in the 39% of consultations, whereas the recommendations of the automated system were more risk averse.…”

Section: Introductionmentioning

confidence: 99%

A probabilistic algorithm with user feedback loop for decision making during the hospital triage process

Zikos

Vandeliwala

Makedon

2014

Proceedings of the 7th International Conference on PErvasive Technologies Related to Assistive Environments

View full text Add to dashboard Cite

In this paper, we describe a probabilistic algorithm with user feedback loop, which can be used for decision making during the patient triage process. Given an R{x, y} the method relies on the user defining a set of x values (i.e. symptoms) and the algorithm returns a collection of y values as a hidden layer (possible diseases), taking into consideration a possible false negative user reporting, by looking into candidate values of y and identifying x values (symptoms) which have not been initially provided by the user. The user can specify parameters such as the minimum probability ratio of the final output, the minimum probability ratio of the y values for which the non-user given x values will be reevaluated, and the maximum number of user feedback loops. In order to validate the method, we use a comprehensive 2012 Medicare Claims dataset with 15 million cases.

show abstract

A study of automated self-assessment in a primary care student health centre setting

Cited by 31 publications

References 11 publications

The quality of diagnosis and triage advice provided by free online symptom checkers and apps in Australia

The quality of diagnosis and triage advice provided by free online symptom checkers and apps in Australia

Digital and online symptom checkers and assessment services for urgent care to inform a new digital platform: a systematic review

A probabilistic algorithm with user feedback loop for decision making during the hospital triage process

Contact Info

Product

Resources

About