The Role of Engagement in a Tailored Web-Based Smoking Cessation Program: Randomized Controlled Trial

Abstract

Background Web-based programs for health promotion, disease prevention, and disease management often experience high rates of attrition. There are 3 questions which are particularly relevant to this issue. First, does engagement with program content predict long-term outcomes? Second, which users are most likely to drop out or disengage from the program? Third, do particular intervention strategies enhance engagement?Objective To determine: (1) whether engagement (defined by the number of Web sections opened) in a Web-based smoking cessation intervention predicts 6-month abstinence, (2) whether particular sociodemographic and psychographic groups are more likely to have lower engagement, and (3) whether particular components of a Web-based smoking cessation program influence engagement.Methods A randomized trial of 1866 smokers was used to examine the efficacy of 5 different treatment components of a Web-based smoking cessation intervention. The components were: high- versus low-personalized message source, high- versus low-tailored outcome expectation, efficacy expectation, and success story messages. Moreover, the timing of exposure to these sections was manipulated, with participants randomized to either a single unified Web program with all sections available at once, or sequential exposure to each section over a 5-week period of time. Participants from 2 large health plans enrolled to receive the online behavioral smoking cessation program and a free course of nicotine replacement therapy (patch). The program included: an introduction section, a section focusing on outcome expectations, 2 sections focusing on efficacy expectations, and a section with a narrative success story (5 sections altogether, each with multiple screens). Most of the analyses were conducted with a stratification of the 2 exposure types. Measures included: sociodemographic and psychosocial characteristics, Web sections opened, perceived message relevance, and smoking cessation 6-months following quit date.Results The total number of Web sections opened was related to subsequent smoking cessation. Participants who were younger, were male, or had less formal education were more likely to disengage from the Web-based cessation program, particularly when the program sections were delivered sequentially over time. More personalized source and high-depth tailored self-efficacy components were related to a greater number of Web sections opened. A path analysis model suggested that the impact of high-depth message tailoring on engagement in the sequentially delivered Web program was mediated by perceived message relevance.Conclusions Results of this study suggest that one of the mechanisms underlying the impact of Web-based smoking cessation interventions is engagement with the program. The source of the message, the degree of message tailoring, and the timing of exposure appear to influence Web-based program engagement.

Understanding and Promoting Effective Engagement With Digital Behavior Change Interventions

“…First, the evidence that usage is associated with intended outcomes is mixed, and largely correlational. [21][22][23] It is difficult to determine to what extent usage mediates behavioral and healthrelated outcomes, as this may be confounded by common factors such as higher motivation and selfregulation skills. Usage metrics also reveal little about offline engagement with intervention content, which is important in inter-ventions that require homework outside the context of the digital intervention.…”

“…23 DBCIs have the potential to generate data sets sufficiently large to be able to reliably model and experimentally test 34 mediation of outcomes by engagement with particular intervention components and to statistically control for confounding moderator effects such as baseline motiva-tion levels. 22,26,35,36 Importantly, usage metrics can be collated with data on users' behavior collected by smartphone sensors, such as movement or location. 37 However, more studies are needed to establish what features or correlates of engagement sensor data can capture reliably, and new statistical approaches will be required to analyze these large and complex data sets.…”

Internet and Telephone Treatment for Smoking Cessation: Mediators and Moderators of Short-Term Abstinence

“…Self-reported smoking status is a commonly accepted outcome measure in Internet cessation trials, 12,[22][23][24][25][26][27] where biochemical verification of abstinence is not feasible and misreporting of abstinence is expected to be minimal given low demand characteristics.…”

Trajectories of 12-Month Usage Patterns for Two Smoking Cessation Websites: Exploring How Users Engage Over Time (Preprint)

“…Knowing these baseline characteristics might allow researchers and intervention designers to tailor e-health interventions to users' unique challenges, needs, and limitations. While studies have found that being a woman, being older, and having a higher education are generally consistent predictors of greater e-health intervention utilization [14][15][16][17], very little is known about the user characteristics that are associated with different patterns of use over time. To our knowledge, only one study has examined this question [12], and found that being female and having higher baseline motivation were associated with more consistent login trajectories.…”