In Germany, the current COVID-19 cases are managed and reported by the local health authorities. The workload of their employees during the pandemic is high, especially in periods of high infection numbers. In this work a decision support toolkit for local health authorities is introduced. A demonstrator web application was developed with the R Shiny framework and is publicly accessible online. It contains five separate tools based on statistical models for specific use cases and corresponding questions of COVID-19 cases and their contacts. The underlying statistical methods have been implemented in a new open-source R package. The toolkit has the potential to support local health authorities’ employees in their daily work. A simulated-based validation of the statistical models and a usability evaluation of the demonstrator application in a user study will be carried out in the future.
Background During the COVID-19 pandemic, local health authorities were responsible for managing and reporting current cases in Germany. Since March 2020, employees had to contain the spread of COVID-19 by monitoring and contacting infected persons as well as tracing their contacts. In the EsteR project, we implemented existing and newly developed statistical models as decision support tools to assist in the work of the local health authorities. Objective The main goal of this study was to validate the EsteR toolkit in two complementary ways: first, investigating the stability of the answers provided by our statistical tools regarding model parameters in the back end and, second, evaluating the usability and applicability of our web application in the front end by test users. Methods For model stability assessment, a sensitivity analysis was carried out for all 5 developed statistical models. The default parameters of our models as well as the test ranges of the model parameters were based on a previous literature review on COVID-19 properties. The obtained answers resulting from different parameters were compared using dissimilarity metrics and visualized using contour plots. In addition, the parameter ranges of general model stability were identified. For the usability evaluation of the web application, cognitive walk-throughs and focus group interviews were conducted with 6 containment scouts located at 2 different local health authorities. They were first asked to complete small tasks with the tools and then express their general impressions of the web application. Results The simulation results showed that some statistical models were more sensitive to changes in their parameters than others. For each of the single-person use cases, we determined an area where the respective model could be rated as stable. In contrast, the results of the group use cases highly depended on the user inputs, and thus, no area of parameters with general model stability could be identified. We have also provided a detailed simulation report of the sensitivity analysis. In the user evaluation, the cognitive walk-throughs and focus group interviews revealed that the user interface needed to be simplified and more information was necessary as guidance. In general, the testers rated the web application as helpful, especially for new employees. Conclusions This evaluation study allowed us to refine the EsteR toolkit. Using the sensitivity analysis, we identified suitable model parameters and analyzed how stable the statistical models were in terms of changes in their parameters. Furthermore, the front end of the web application was improved with the results of the conducted cognitive walk-throughs and focus group interviews regarding its user-friendliness.
BACKGROUND In the COVID-19 pandemic, local health authorities are responsible for managing and reporting the current cases in Germany. Since March 2020, the employees had to focus on the pandemic. In the EsteR project, we implemented existing and newly developed statistical models in decision support tools to assist the work in the local health authorities. OBJECTIVE The two main goals of this work are investigating the stability of the answers provided by our statistical tools regarding model parameters and evaluating the usability and applicability of our web application. METHODS For the model stability assessment, a sensitivity analysis was carried out for all five statistical models. The default parameters of our models and the ranges the parameters were tested in were based on a prior literature review on COVID-19 properties. For the usability evaluation of the web application, cognitive walkthroughs and focus group interviews were conducted with employees of two different local health authorities. RESULTS The simulation results showed that some statistical models are more sensitive to changes in their parameters than others. For each of the single person use cases we were able to find an area where we rate the respective model to be stable, whereas for the group use cases the stability highly depends on the user inputs. The cognitive walkthroughs and the focus group interview disclosed that the user interface had to be simplified and more information was needed as guidance. In general, the testers rated the web application as helpful, especially for new employees. CONCLUSIONS This evaluation study allowed for a refinement of our toolkit. With the simulations, we identified suitable model parameters and analyzed how stable the statistical models are regarding changes in their parameters. Furthermore, the web application was improved with the results of the cognitive walkthroughs and focus group interviews.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.