Engagement With a Behavior Change App for Alcohol Reduction: Data Visualization for Longitudinal Observational Study

Abstract

Background Behavior change apps can develop iteratively, where the app evolves into a complex, dynamic, or personalized intervention through cycles of research, development, and implementation. Understanding how existing users engage with an app (eg, frequency, amount, depth, and duration of use) can help guide further incremental improvements. We aim to explore how simple visualizations can provide a good understanding of temporal patterns of engagement, as usage data are often longitudinal and rich. Objective This study aims to visualize behavioral engagement with Drink Less, a behavior change app to help reduce hazardous and harmful alcohol consumption in the general adult population of the United Kingdom. Methods We explored behavioral engagement among 19,233 existing users of Drink Less. Users were included in the sample if they were from the United Kingdom; were 18 years or older; were interested in reducing their alcohol consumption; had a baseline Alcohol Use Disorders Identification Test score of 8 or above, indicative of excessive drinking; and had downloaded the app between May 17, 2017, and January 22, 2019 (615 days). Measures of when sessions begin, length of sessions, time to disengagement, and patterns of use were visualized with heat maps, timeline plots, k-modes clustering analyses, and Kaplan-Meier plots. Results The daily 11 AM notification is strongly associated with a change in engagement in the following hour; reduction in behavioral engagement over time, with 50.00% (9617/19,233) of users disengaging (defined as no use for 7 or more consecutive days) 22 days after download; identification of 3 distinct trajectories of use, namely engagers (4651/19,233, 24.18% of users), slow disengagers (3679/19,233, 19.13% of users), and fast disengagers (10,903/19,233, 56.68% of users); and limited depth of engagement with 85.076% (7,095,348/8,340,005) of screen views occurring within the Self-monitoring and Feedback module. In addition, a peak of both frequency and amount of time spent per session was observed in the evenings. Conclusions Visualizations play an important role in understanding engagement with behavior change apps. Here, we discuss how simple visualizations helped identify important patterns of engagement with Drink Less. Our visualizations of behavioral engagement suggest that the daily notification substantially impacts engagement. Furthermore, the visualizations suggest that a fixed notification policy can be effective for maintaining engagement for some users but ineffective for others. We conclude that optimizing the notification policy to target both effectiveness and engagement is a worthwhile investment. Our future goal is to both understand the causal effect of the notification on engagement and further optimize the notification policy within Drink Less by tailoring to contextual circumstances of individuals over time. Such tailoring will be informed from the findings of our micro-randomized trial (MRT), and these visualizations were useful in both gaining a better understanding of engagement and designing the MRT.

ProHealth eCoach: User-Centered Design and Development of an eCoach App to Promote Healthy Lifestyle with Personalized Activity Recommendations

“…Their research reveals the importance of a user-centered/patient-centered approach in achieving user engagement (or user evaluation) to enhance the effectiveness of behavioral interventions. Bell et al [32] explored how user engagement can improve the research and development of a behavior change app iteratively. They performed a behavioral engagement longitudinal observational study on a group of participants using their "Drink Less" behavior change app for alcohol reduction and explored parameters, such as frequency, amount, depth, and duration of the study with a simple data visualization tool to understand user psychology to improve the app further.…”

Refining the content and design of an alcohol reduction app, Drink Less, to improve its usability and effectiveness: a mixed methods approach

“…Response: A previous study investigated engagement with the original version of the app and we have now included this in the discussion on page 27 along with highlighting that future research should consider comparing engagement between the original and refined version of the app: "A number of major changes were made to the original version of the app meaning that the overall content of the original and refined versions of the app are not directly comparable. A previous study has investigated engagement with the original version of the app (Bell et al, 2020) though no research has yet been conducted on engagement with the refined version of the app. It is an empirical question as to whether engagement with the app has changed as a result of the refined version of the app being released and a question to consider for future research.…”

Contrasting a Mobile App With a Conversational Chatbot for Reducing Alcohol Consumption: Randomized Controlled Pilot Trial

“…One important advantage of technology-based alcohol interventions is that they can be accessed from home and the intervention is situated within daily contexts [ 27 - 29 ]. Much of what leads to continued alcohol use or relapse occurs in an individual’s everyday environment, where, unlike a traditional treatment context, exposure to alcohol-related cues, stress, and negative affect are uncontrolled [ 30 , 31 ]. These contextual and situational cues may overwhelm an individual’s coping resources and other skills that were discussed during treatment sessions with the hope that they would be used when facing cravings for alcohol triggered by internal and external factors encountered in daily life [ 31 , 32 ].…”

“…In a recent systematic review of health behavior change apps, including diet, physical activity, and alcohol use, many reported improvements in targeted outcomes, but few demonstrated significant treatment effects over comparison groups in randomized trials [ 34 ]. The reported limitations included study design issues (eg, nonrepresentative samples and inadequate comparison groups), intervention design issues (eg, features offered are not based on theory or existing evidence), and limited or short-lived engagement with the app [ 30 , 34 ].…”

“…An advantage of apps is their ability to generate utilization data, which are commonly used as a proxy for quantifying engagement. Utilization metrics typically include the number of log-ins, proportion of features accessed, and frequency and duration of use [ 30 ]. However, interpreting the relative importance of these metrics to determine the amount and type of engagement that is sufficient to achieve the desired behavioral and health outcomes (ie, effective engagement) has not been established [ 37 ].…”