Participant Engagement in Microrandomized Trials of mHealth Interventions: Scoping Review

Abstract

Background Microrandomized trials (MRTs) have emerged as the gold standard for the development and evaluation of multicomponent, adaptive mobile health (mHealth) interventions. However, not much is known about the state of participant engagement measurement in MRTs of mHealth interventions. Objective In this scoping review, we aimed to quantify the proportion of existing or planned MRTs of mHealth interventions to date that have assessed (or have planned to assess) engagement. In addition, for the trials that have explicitly assessed (or have planned to assess) engagement, we aimed to investigate how engagement has been operationalized and to identify the factors that have been studied as determinants of engagement in MRTs of mHealth interventions. Methods We conducted a broad search for MRTs of mHealth interventions in 5 databases and manually searched preprint servers and trial registries. Study characteristics of each included evidence source were extracted. We coded and categorized these data to identify how engagement has been operationalized and which determinants, moderators, and covariates have been assessed in existing MRTs. Results Our database and manual search yielded 22 eligible evidence sources. Most of these studies (14/22, 64%) were designed to evaluate the effects of intervention components. The median sample size of the included MRTs was 110.5. At least 1 explicit measure of engagement was included in 91% (20/22) of the included MRTs. We found that objective measures such as system usage data (16/20, 80%) and sensor data (7/20, 35%) are the most common methods of measuring engagement. All studies included at least 1 measure of the physical facet of engagement, but the affective and cognitive facets of engagement have largely been neglected (only measured by 1 study each). Most studies measured engagement with the mHealth intervention (Little e) and not with the health behavior of interest (Big E). Only 6 (30%) of the 20 studies that measured engagement assessed the determinants of engagement in MRTs of mHealth interventions; notification-related variables were the most common determinants of engagement assessed (4/6, 67% studies). Of the 6 studies, 3 (50%) examined the moderators of participant engagement—2 studies investigated time-related moderators exclusively, and 1 study planned to investigate a comprehensive set of physiological and psychosocial moderators in addition to time-related moderators. Conclusions Although the measurement of participant engagement in MRTs of mHealth interventions is prevalent, there is a need for future trials to diversify the measurement of engagement. There is also a need for researchers to address the lack of attention to how engagement is determined and moderated. We hope that by mapping the state of engagement measurement in existing MRTs of mHealth interventions, this review will encourage researchers to pay more attention to these issues when planning for engagement measurement in future trials.

Population-Based Cancer Prevention Education Intervention Through mHealth: A Randomized Controlled Trial

“…In the app group, this may be explained by the fact that the “push” notification needed to be activated by the user to acknowledge reception of the messages; also, some users may have uninstalled the app during the study, without notifying the interviewers at the post-intervention survey. To overcome this, system usage data should be enhanced to quantitatively measure engagement with mHealth interventions [ 32 ].…”

Engagement with mHealth Alcohol Interventions: User Perspectives on an App or Chatbot-Delivered Program to Reduce Drinking

“…Engagement is a growing concept in research, given its association with intervention outcomes; however, it can be measured in various ways, and researchers have encouraged the priority of improving engagement measures [ 34 ]. While our study explored facilitators and barriers to utilization to explain differences in utilization of Step Away [ 35 ], focusing on utilization alone is likely insufficient to determine whether the skills learned through the intervention were incorporated into the user’s everyday “offline” life [ 34 ]. This may be measured by qualitative data regarding the user’s application of the skills learned and how they used advice from the app, as well as data collected within the app on how users were able to apply these skills.…”