Syndromic surveillance insights from a symptom assessment app before and during COVID-19 measures in Germany and the United Kingdom: results from repeated cross-sectional analyses
Abstract:Background: Unprecedented lockdown measures have been introduced in countries across the world to mitigate the spread and consequences of COVID-19. While attention has focused on the effects of these measures on epidemiological indicators relating directly to the infection, there is increased recognition of their broader health implications. However, assessing these implications in real time is a challenge, due to limitations of existing syndromic surveillance data and tools.
Objective: To explore the added va… Show more
“…CDSS facilitating clinical decision-making have exhibited strong performance in several previous studies, including the Ada-App 8,9,19–23. Gilbert et al10 recently evaluated the performance of eight digital symptom assessment apps, showing that Ada achieved a higher accuracy (71%) in comparison to other apps for the top 3 diagnoses (average: 47%), providing safe urgency advice in 97%.…”
Objective: To evaluate the diagnostic accuracy of the app-based diagnostic tool Ada and the impact on patient outcome in the emergency room (ER). Background: Artificial intelligence-based diagnostic tools can improve targeted processes in health care delivery by integrating patient information with a medical knowledge base and a machine learning system, providing clinicians with differential diagnoses and recommendations. Methods: Patients presenting to the ER with abdominal pain self-assessed their symptoms using the Ada-App under supervision and were subsequently assessed by the ER physician. Diagnostic accuracy was evaluated by comparing the App-diagnoses with the final discharge diagnoses. Timing of diagnosis and time to treatment were correlated with complications, overall survival, and length of hospital stay. Results: In this prospective, double-blinded study, 450 patients were enrolled and followed up until day 90. Ada suggested the final discharge diagnosis in 52.0% (95% CI [0.47, 0.57]) of patients compared with the classic doctor-patient interaction, which was significantly superior with 80.9% (95% CI [0.77, 0.84], P < 0.001). However, when diagnostic accuracy of both were assessed together, Ada significantly increased the accuracy rate (87.3%, P < 0.001), when compared with the ER physician alone. Patients with an early time point of diagnosis and rapid treatment allocation exhibited significantly reduced complications (P < 0.001) and length of hospital stay (P < 0.001). Conclusion: Currently, the classic patient-physician interaction is superior to an AI-based diagnostic tool applied by patients. However, AI tools have the potential to additionally benefit the diagnostic efficacy of clinicians and improve quality of care.
“…CDSS facilitating clinical decision-making have exhibited strong performance in several previous studies, including the Ada-App 8,9,19–23. Gilbert et al10 recently evaluated the performance of eight digital symptom assessment apps, showing that Ada achieved a higher accuracy (71%) in comparison to other apps for the top 3 diagnoses (average: 47%), providing safe urgency advice in 97%.…”
Objective: To evaluate the diagnostic accuracy of the app-based diagnostic tool Ada and the impact on patient outcome in the emergency room (ER). Background: Artificial intelligence-based diagnostic tools can improve targeted processes in health care delivery by integrating patient information with a medical knowledge base and a machine learning system, providing clinicians with differential diagnoses and recommendations. Methods: Patients presenting to the ER with abdominal pain self-assessed their symptoms using the Ada-App under supervision and were subsequently assessed by the ER physician. Diagnostic accuracy was evaluated by comparing the App-diagnoses with the final discharge diagnoses. Timing of diagnosis and time to treatment were correlated with complications, overall survival, and length of hospital stay. Results: In this prospective, double-blinded study, 450 patients were enrolled and followed up until day 90. Ada suggested the final discharge diagnosis in 52.0% (95% CI [0.47, 0.57]) of patients compared with the classic doctor-patient interaction, which was significantly superior with 80.9% (95% CI [0.77, 0.84], P < 0.001). However, when diagnostic accuracy of both were assessed together, Ada significantly increased the accuracy rate (87.3%, P < 0.001), when compared with the ER physician alone. Patients with an early time point of diagnosis and rapid treatment allocation exhibited significantly reduced complications (P < 0.001) and length of hospital stay (P < 0.001). Conclusion: Currently, the classic patient-physician interaction is superior to an AI-based diagnostic tool applied by patients. However, AI tools have the potential to additionally benefit the diagnostic efficacy of clinicians and improve quality of care.
“…4 Studies in Europe and Asia have found that self-reported symptoms collected through mobile applications had strong spatial correlations with confirmed COVID-19 cases 5 and that by collecting data before and after COVID-19 restrictions, the tool was effective in evaluating control measures. 6 In April 2020, C19Check.com (C19Check) was launched by Emory University and Vital Software Inc. in Atlanta, Georgia (GA). The online symptom tracker, freely available in 31 languages, prompts users to report their symptoms and then generates evidence-based summaries of risk of COVID-19 infection and advice for seeking healthcare.…”
Section: Discussionmentioning
confidence: 99%
“…The amount of error in the forecast is likely because C19Check use itself is not a cause of the surges and declines in cases and hospitalizations. The significant results for all time-lags tested indicate that our findings on the predictability of C19Check use were not impacted by time-lags, demonstrating the effectiveness of C19Check as a tool for syndromic surveillance of COVID-19 cases and hospitalizations in GA. Other real-time syndromic surveillance tools have been used to detect early signals, monitor population transmission dynamics and identify hotspots in different countries 5,6,9,10 and various regions of the US. 9,11 However, we also evaluated the performance of an internet-based self-triage tool in predicting COVID-19 cases and hospitalizations.…”
Introduction: The coronavirus 2019 (COVID-19) pandemic has created significant burden on healthcare systems throughout the world. Syndromic surveillance, which collects real-time data based on a range of symptoms rather than laboratory diagnoses, can help provide timely information in emergency response. We examined the effectiveness of a web-based COVID-19 symptom checking tool (C19Check) in the state of Georgia (GA) in predicting COVID-19 cases and hospitalizations.
Methods: We analyzed C19Check use data, COVID-19 cases, and hospitalizations from April 22– November 28, 2020. Cases and hospitalizations in GA were extracted from the Georgia Department of Public Health data repository. We used the Granger causality test to assess whether including C19Check data can improve predictions compared to using previous COVID-19 cases and hospitalizations data alone. Vector autoregression (VAR) models were fitted to forecast cases and hospitalizations from November 29 - December 12, 2020. We calculated mean absolute percentage error to estimate the errors in forecast of cases and hospitalizations.
Results: There were 25,861 C19Check uses in GA from April 22–November 28, 2020. Time-lags tested in Granger causality test for cases (6-8 days) and hospitalizations (10-12 days) were significant (P= <0.05); the mean absolute percentage error of fitted VAR models were 39.63% and 15.86%, respectively.
Conclusion: The C19Check tool was able to help predict COVID-19 cases and related hospitalizations in GA. In settings where laboratory tests are limited, a real-time, symptom-based assessment tool can provide timely and inexpensive data for syndromic surveillance to guide pandemic response. Findings from this study demonstrate that online symptom-checking tools can be a source of data for syndromic surveillance, and the data may help improve predictions of cases and hospitalizations.
“…While participatory crowdsourced syndromic surveillance has been utilized in many contexts [1]–[7], including for COVID-19 [8], [9], their ability to track an emerging outbreak at a high spatial resolution has not been evaluated previously. Here, we show that one such system, though noisy, provides an indication of where and when to expect new cases, suggesting that it could be a useful model in other places that need to map COVID-19 risk for decision making.…”
Limitations in laboratory diagnostic capacity and reporting delays have hampered efforts to mitigate and control the ongoing coronavirus disease 2019 (COVID-19) pandemic globally. To augment traditional lab and hospital-based surveillance, Bangladesh established a participatory surveillance system for the public to self-report symptoms consistent with COVID-19 through multiple channels. Here, we report on the use of this system, which received over 3 million responses within two months, for tracking the COVID-19 outbreak in Bangladesh. Although we observe considerable noise in the data and initial volatility in the use of the different reporting mechanisms, the self-reported syndromic data exhibits a strong association with lab-confirmed cases at a local scale. Moreover, the syndromic data also suggests an earlier spread of the outbreak across Bangladesh than is evident from the confirmed case counts, consistent with predicted spread of the outbreak based on population mobility data. Our results highlight the usefulness of participatory syndromic surveillance for mapping disease burden generally, and particularly during the initial phases of an emerging outbreak.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.