Participant Engagement in Microrandomized Trials of mHealth Interventions: Scoping Review

Abstract

Background: Micro-randomised trials (MRTs) have emerged as the gold standard for the development and evaluation of multi-component, adaptive mobile health (mHealth) interventions. However, not much is known about the state of participant engagement measurement in MRTs of mHealth interventions.Objective: In this review, we aimed to quantify the proportion of existing/planned MRTs of mHealth interventions to date that have assessed (or have planned to assess) engagement. For trials that have assessed (or have planned to assess) engagement, we also aimed to investigate how engagement has been operationalized and to identify what kind of factors have been studied as determinants of engagement in MRTs of mHealth interventions.Methods: We conducted a broad search for MRTs of mHealth interventions in 5 databases and manually searched preprint servers and trial registries. Study characteristics of each included evidence source were extracted. We coded and categorized this data to identify how engagement has been operationalized and which determinants, moderators, and control variables have been assessed in existing MRTs.Results: Our database and manual searches yielded 22 eligible evidence sources. Most of these studies were designed to evaluate the effect of intervention components on health outcomes (14/22 studies). The median sample size of the included MRTs was 110.5. At least one measure of engagement was included in 90.91% (20/22 studies) of the included MRTs. Engagement operationalized as usage (16/20; 80%) was most prevalent followed by responsiveness (11/20; 55%), practice (6/20; 30%), and then comprehension (1/20; 5%). Only 6 of the 20 studies to measure engagement (30%) assessed the determinants of engagement in MRTs of mHealth interventions – notification-related variables were the most common determinants of engagement assessed (4/6 studies). Three of these studies also examined the moderators of participant engagement – 2 studies investigated time-related moderators exclusively and 1 study planned to investigate a comprehensive set of physiological and psychosocial moderators in addition to time-related moderators.Conclusions: Although the measurement of participant engagement in MRTs of mHealth interventions is prevalent, there is a need for future trials to strike a balance between measuring engagement as usage/responsiveness and as practice. There is also a need for researchers to address the lack of attention to how engagement is determined and moderated in MRTs of mHealth interventions. We hope that by mapping the state of engagement measurement in existing MRTs of mHealth interventions, this review will encourage researchers to pay more attention to these issues when planning for engagement measurement in future trials.

Participant Engagement in Microrandomized Trials of mHealth Interventions: Scoping Review

“…The protocol for this scoping review was developed using the Joanna Briggs Institute Manual for Evidence Synthesis [ 19 ] and was designed to ensure adherence to the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analysis extension for Scoping Reviews) guidelines [ 20 ]. The protocol and its appendices were prospectively registered with the Open Science Framework (OSF) on June 30, 2022 [ 21 ].…”

“…We prioritized the inclusion of papers published in peer-reviewed journals. We included preprints, trial protocols, and dissertations (this was mistakenly left out of the “Types of Sources” section of our protocol [ 21 ]) only if no corresponding peer-reviewed journal articles were available. Conference abstracts were excluded from this scoping review.…”

“…Therefore, we condensed the 2 planned searches into 1 by using the Boolean operator OR instead, such that our database searches indexed any MRTs that mentioned mHealth interventions or engagement-related terms. The comprehensive search strategies for all 5 databases (and their respective previous iterations) can be found on OSF [ 21 ].…”

“…As described in our protocol [ 21 ], we developed an initial data extraction form (a Microsoft Excel [Microsoft Corporation] spreadsheet) to chart the data from eligible evidence sources to obtain the information necessary to answer our review questions. Both authors (UL and BC) piloted this initial data extraction form with 4 included articles to calibrate the charting process and to ensure that relevant data items were captured by the form.…”

“…After piloting the form and during the charting process, we included additional data items to capture the following information: primary and secondary (if any) outcomes of the study, randomization design of the MRT, frequency of microrandomization, and the overall duration of the MRT. The final version of the data extraction form is available on OSF [ 21 ].…”