Digital Contact Tracing Apps for COVID-19: Development of a Citizen-Centered Evaluation Framework

Abstract

Background The silent transmission of COVID-19 has led to an exponential growth of fatal infections. With over 4 million deaths worldwide, the need to control and stem transmission has never been more critical. New COVID-19 vaccines offer hope. However, administration timelines, long-term protection, and effectiveness against potential variants are still unknown. In this context, contact tracing and digital contact tracing apps (CTAs) continue to offer a mechanism to help contain transmission, keep people safe, and help kickstart economies. However, CTAs must address a wide range of often conflicting concerns, which make their development/evolution complex. For example, the app must preserve citizens’ privacy while gleaning their close contacts and as much epidemiological information as possible. Objective In this study, we derived a compare-and-contrast evaluative framework for CTAs that integrates and expands upon existing works in this domain, with a particular focus on citizen adoption; we call this framework the Citizen-Focused Compare-and-Contrast Evaluation Framework (C3EF) for CTAs. Methods The framework was derived using an iterative approach. First, we reviewed the literature on CTAs and mobile health app evaluations, from which we derived a preliminary set of attributes and organizing pillars. These attributes and the probing questions that we formulated were iteratively validated, augmented, and refined by applying the provisional framework against a selection of CTAs. Each framework pillar was then subjected to internal cross-team scrutiny, where domain experts cross-checked sufficiency, relevancy, specificity, and nonredundancy of the attributes, and their organization in pillars. The consolidated framework was further validated on the selected CTAs to create a finalized version of C3EF for CTAs, which we offer in this paper. Results The final framework presents seven pillars exploring issues related to CTA design, adoption, and use: (General) Characteristics, Usability, Data Protection, Effectiveness, Transparency, Technical Performance, and Citizen Autonomy. The pillars encompass attributes, subattributes, and a set of illustrative questions (with associated example answers) to support app design, evaluation, and evolution. An online version of the framework has been made available to developers, health authorities, and others interested in assessing CTAs. Conclusions Our CTA framework provides a holistic compare-and-contrast tool that supports the work of decision-makers in the development and evolution of CTAs for citizens. This framework supports reflection on design decisions to better understand and optimize the design compromises in play when evolving current CTAs for increased public adoption. We intend this framework to serve as a foundation for other researchers to build on and extend as the technology matures and new CTAs become available.

Pathways, technology and the patient—connected health through the lifecycle

“…This directive requires that all public sector bodies in the EU have accessible online websites and mobile apps, through which many connected health solutions are implemented. In their research, Tsvyatkova et al ( 60 ), present accessibility as having concern for the quality of being “easy to reach and use”.…”

Evaluating the Added Value of Digital Contact Tracing Support Tools for Citizens: Framework Development

“…Frameworks about how to evaluate proximity-based apps and the effectiveness of CT in general have been published already [29][30][31], but none, to the best of our knowledge, have been published about how to assess the effectiveness of DCTS tools. This paper presents a framework for how DCTS tools can be assessed when compared with traditional CT, with regard to various important factors that are necessary to curb the transmission of infectious diseases, such as increasing the speed and comprehensiveness of the CT process and relieving the burden on PHPs.…”

Preferences in the Intention to Download a COVID Tracing App: A Discrete Choice Experiment Study in the Netherlands and Turkey

“…The four attributes (data protection, manufacturer, reward, and gaming) and levels selected for inclusion in the DCE are shown in Table 1. The factors have been based on a mix between a literature review (Tsvyatkova et al, 2022) and elements (reward [no reward vs. a voucher as a reward] and gaming [no gaming elements vs. gaming elements]) that were considered relevant for next steps in the Horizon2020 project PandeVITA. 1…”

“…Nonetheless, the actual adoption and usage of mobile health apps is rather low and lagging behind its promises (Walrave et al, 2020). Recent studies have started to better understand how CTA's can address a wide range of conflicting elements, such as preserving citizens' privacy while gleaning close contacts and epidemiological information as possible (Tsvyatkova et al, 2022), but there is still limited evidence supporting adoption or willingness to download preferences for CTA's. It is therefore needed to better understand which attributes affect people's willingness to download a CTA (Anastasiadou et al, 2018a,b).…”