Evaluation of Digital Interventions for Physical Activity Promotion: Scoping Review

Abstract

Background Digital interventions are interventions supported by digital tools or technologies, such as mobile apps, wearables, or web-based software. Digital interventions in the context of public health are specifically designed to promote and improve health. Recent reviews have shown that many digital interventions target physical activity promotion; however, it is unclear how such digital interventions are evaluated. Objective We aimed to investigate evaluation strategies in the context of digital interventions for physical activity promotion using a scoping review of published reviews. We focused on the target (ie, user outcomes or tool performance), methods (ie, tool data or self-reported data), and theoretical frameworks of the evaluation strategies. Methods A protocol for this study was preregistered and published. From among 300 reviews published up to March 19, 2021 in Medline, PsycINFO, and CINAHL databases, 40 reviews (1 rapid, 9 scoping, and 30 systematic) were included in this scoping review. Two authors independently performed study selection and data coding. Consensus was reached by discussion. If applicable, data were coded quantitatively into predefined categories or qualitatively using definitions or author statements from the included reviews. Data were analyzed using either descriptive statistics, for quantitative data (relative frequencies out of all studies), or narrative synthesis focusing on common themes, for qualitative data. Results Most reviews that were included in our scoping review were published in the period from 2019 to 2021 and originated from Europe or Australia. Most primary studies cited in the reviews included adult populations in clinical or nonclinical settings, and focused on mobile apps or wearables for physical activity promotion. The evaluation target was a user outcome (efficacy, acceptability, usability, feasibility, or engagement) in 38 of the 40 reviews or tool performance in 24 of the 40 reviews. Evaluation methods relied upon objective tool data (in 35/40 reviews) or other data from self-reports or assessments (in 28/40 reviews). Evaluation frameworks based on behavior change theory, including goal setting, self-monitoring, feedback on behavior, and educational or motivational content, were mentioned in 22 out of 40 reviews. Behavior change theory was included in the development phases of digital interventions according to the findings of 20 out of 22 reviews. Conclusions The evaluation of digital interventions is a high priority according to the reviews included in this scoping review. Evaluations of digital interventions, including mobile apps or wearables for physical activity promotion, typically target user outcomes and rely upon objective tool data. Behavior change theory may provide useful guidance not only for development of digital interventions but also for the evaluation of user outcomes in the context of physical activity promotion. Future research should investigate factors that could improve the efficacy of digital interventions and the standardization of terminology and reporting in this field. International Registered Report Identifier (IRRID) RR2-10.2196/35332

Digital Technologies for Health Promotion and Disease Prevention in Older People: Protocol for a Scoping Review

“…The development and reporting of the search strategy adheres to PRESS (Peer Review of Electronic Search Strategies) [34] and PRISMA-S (PRISMA Statement for Reporting Literature Searches in Systematic Reviews) [35] guidelines. The preliminary search syntax was developed based on our own syntax from a scoping review on the evaluation of digital interventions for physical activity promotion [17,36] and the search strategies reported in other relevant reviews (Table 1). According to the PRESS guideline [34], we first adapted the preliminary syntax to match our PCC criteria (Textbox 1) as follows: (1) digital technologies, (2) older people, and (3) health promotion and disease prevention.…”

“…In the first step, 2 items on AMSTAR2 (item 2: presence of a review protocol and item 7: presence of a list of excluded studies) will be rated to identify any systematic reviews with critically low confidence ratings. These 2 items were chosen because they are typically not fulfilled in systematic reviews of nondigital or digital health interventions [17,33,41]. In the second step, any systematic reviews that fulfill item 2, item 7, or both will be rated with all 16 AMSTAR2 items according to AMSTAR2 guidance [40].…”

“…In general, co-creation and feedback from older users are required to develop appropriate digital technologies for healthy aging in this target population [13][14][15]. Furthermore, evaluation of cost-effectiveness [16] and user outcomes in the context of health promotion and disease prevention [17] is required to better understand if and how digital technologies work.…”

“…Finally, the most common health target of digital interventions for health promotion and disease prevention is physical activity, according to a scoping review [20] and recent systematic reviews that assessed the efficacy of such interventions in older people (Table 1). As illustrated in Table 1 and discussed in other scoping reviews [17,21], the terminology in the field of digital health promotion and disease prevention is highly diverse, and a uniform definition of the age of older people does not exist. Therefore, a new scoping review is required to more broadly identify any available digital technologies that target any aspects of healthy aging in nonclinical settings (ie, daily life).…”

“…This approach will assure that the relevant literature will be identified either in our literature search or in other reviews. Furthermore, identification of other reviews in this rapidly developing field can potentially reduce research waste that occurs when new reviews are produced, although reviews on similar topics already exist [17,33].…”