Understanding Health Behavior Technology Engagement: Pathway to Measuring Digital Behavior Change Interventions

Abstract

Researchers and practitioners of digital behavior change interventions (DBCI) use varying and, often, incongruent definitions of the term “engagement,” thus leading to a lack of precision in DBCI measurement and evaluation. The objective of this paper is to propose discrete definitions for various types of user engagement and to explain why precision in the measurement of these engagement types is integral to ensuring the intervention is effective for health behavior modulation. Additionally, this paper presents a framework and practical steps for how engagement can be measured in practice and used to inform DBCI design and evaluation. The key purpose of a DBCI is to influence change in a target health behavior of a user, which may ultimately improve a health outcome. Using available literature and practice-based knowledge of DBCI, the framework conceptualizes two primary categories of engagement that must be measured in DBCI. The categories are health behavior engagement, referred to as “Big E,” and DBCI engagement, referred to as “Little e.” DBCI engagement is further bifurcated into two subclasses: (1) user interactions with features of the intervention designed to encourage frequency of use (ie, simple login, games, and social interactions) and make the user experience appealing, and (2) user interactions with behavior change intervention components (ie, behavior change techniques), which influence determinants of health behavior and subsequently influence health behavior. Achievement of Big E in an intervention delivered via digital means is contingent upon Little e. If users do not interact with DBCI features and enjoy the user experience, exposure to behavior change intervention components will be limited and less likely to influence the behavioral determinants that lead to health behavior engagement (Big E). Big E is also dependent upon the quality and relevance of the behavior change intervention components within the solution. Therefore, the combination of user interactions and behavior change intervention components creates Little e, which is, in turn, designed to improve Big E. The proposed framework includes a model to support measurement of DBCI that describes categories of engagement and details how features of Little e produce Big E. This framework can be applied to DBCI to support various health behaviors and outcomes and can be utilized to identify gaps in intervention efficacy and effectiveness.

Beyond clicks and downloads: a call for a more comprehensive approach to measuring mobile-health app engagement

“…2 Usage data is assumed to capture different types and depths of app engagement, yet misses cognitive and emotional responses to the app and is disconnected from behaviour change in real-world settings. 3,4 User engagement has become an umbrella term used to describe a host of conceptually unique user-centred outcomes, including usability, acceptability, feasibility and satisfaction. 5 These different user experiences are all evaluated with the same kinds of usage statistics: dwell time, bounce rates and number of downloads, logins, visits and specific interactions, such as clicking on links, watching videos, completing modules.…”

“…It has been proposed that the field of m-health evaluation could be advanced if we understood the relationship between out-of-app and in-app engagement. 3 However, a barrier to this is the way in which in-app user engagement is currently conceptualised and measured.…”

“…2 Usage data is assumed to capture different types and depths of app engagement, yet misses cognitive and emotional responses to the app and is disconnected from behaviour change in real-world settings. 3,4…”

Use and Effect of Web-Based Embodied Conversational Agents for Improving Eating Behavior and Decreasing Loneliness Among Community-Dwelling Older Adults: Protocol for a Randomized Controlled Trial

“…In order to conceptualize and measure engagement, Cole-Lewis et al [ 19 ] state that it is necessary for users to have appropriate levels of interaction with the technology and that the behavioral change components are relevant. Hence, we present 2 conceptual models.…”

Twitter-Based Social Support Added to Fitbit Self-Monitoring for Decreasing Sedentary Behavior: Protocol for a Randomized Controlled Pilot Trial With Female Patients From a Women’s Heart Clinic

“…Often used as a supplementary aid, the potential for utilizing Twitter as a stand-alone to deliver health behavior interventions is not yet fully realized [ 16 ]. When used, engagement strongly predicts the benefits [ 25 , 26 ]. Tweet2Quit, a Twitter-based intervention for smoking cessation, is among the first successful interventions designed to promote smoking cessation with sustained long-term engagement and maintenance of changed behavior [ 27 , 28 ].…”