Performance of national COVID-19 ‘symptom checkers’: a comparative case simulation study

Abstract

ObjectivesIdentifying those individuals requiring medical care is a basic tenet of the pandemic response. Here, we examine the COVID-19 community triage pathways employed by four nations, specifically comparing the safety and efficacy of national online ‘symptom checkers’ used within the triage pathway.MethodsA simulation study was conducted on current, nationwide, patient-led symptom checkers from four countries (Singapore, Japan, USA and UK). 52 cases were simulated to approximate typical COVID-19 presentations (mild, moderate, severe and critical) and COVID-19 mimickers (eg, sepsis and bacterial pneumonia). The same simulations were applied to each of the four country’s symptom checkers, and the recommendations to refer on for medical care or to stay home were recorded and compared.ResultsThe symptom checkers from Singapore and Japan advised onward healthcare contact for the majority of simulations (88% and 77%, respectively). The USA and UK symptom checkers triaged 38% and 44% of cases to healthcare contact, respectively. Both the US and UK symptom checkers consistently failed to identify severe COVID-19, bacterial pneumonia and sepsis, triaging such cases to stay home.ConclusionOur results suggest that whilst ‘symptom checkers’ may be of use to the healthcare COVID-19 response, there is the potential for such patient-led assessment tools to worsen outcomes by delaying appropriate clinical assessment. The key features of the well-performing symptom checkers are discussed.

Restricted access to the NHS during the COVID-19 pandemic: Is it time to move away from the rationed clinical response?

“…UK-wide, patients were advised to stay at home, book a SARS-CoV-2 test, and if concerned consult either an automated online symptom checker or non-clinical telephone triage system [ 12 , 13 ]. Notably, thresholds for onward referral using these new and unproven triage systems were high [15] . Equally concerning, the subsequent automated safety-net advice given to the patient included ‘how to manage breathlessness at home’ [13] – a practice that would have been inconceivable in 2019.…”

“…Unless the thresholds for onward assessment are set low, many opportunities to prevent disease progression will be missed, and with it the opportunity to reduce mortality, prevent post-pneumonia complications, and prevent prolonged and protracted hospital admissions. In the UK, the thresholds for recommending any clinical contact are set high, and not just for the areas suffering a surge of infections, but for the entire nation - restricting access to care even where healthcare demands are relatively low [15] .…”

“…Typically, in Singapore, any member of the public can book a same day appointment at one of 900 public health clinics (run by primary care), receive a clinical assessment, a SARS-CoV-2 swab, and be triaged home with follow-up, or referred to a dedicated secondary care service for further assessment [ 41 , 42 ]. During a surge of SARS-CoV-2 cases in April 2020, Singapore's online triage tool was changed to advise the non-breathless, young patient to stay at home and to contact their GP if symptoms hadn't improved after three days [15] . While still a compromised response - and one that was quickly reversed when case numbers fell -, it permitted some low-risk relief to the healthcare service.…”

COVID-19 in Brazil—Preliminary Analysis of Response Supported by Artificial Intelligence in Municipalities

“…Symptom checkers, which are self-directed risk assessments tools, has been shown to improve live quality and survival in oncology patients ( 2 ). Its use for COVID-19 is currently increasing and has been used to identify symptomatic patients or potential severe cases with the main objective to triage those individuals who would benefit from clinical assessment and/or management into further care ( 3 ). A recent French study showed that a national self-triage web application contributed to the alleviation of calls to the emergency calls centers and may be used for predicting increasing burden of hospital ( 4 ).…”

Use and Users of the Web-Based Omaolo Covid-19 Symptom Self-Assesment Tool in Finland Since March 16, 2020

“…Symptom checkers are algorithm-based programs that provide potential virtual diagnoses and triage advice [17][18][19]. Symptom checkers promise a cost-effective way of enabling rapidly scalable epidemiological data collection and analysis, with varying diagnostic accuracy [20,21].…”