Comparing usage of a web and app stress management intervention: An observational study

Abstract

Choices in the design and delivery of digital health behaviour interventions may have a direct influence on subsequent usage and engagement. Few studies have been able to make direct, detailed comparisons of differences in usage between interventions that are delivered via web or app. This study compared the usage of two versions of a digital stress management intervention, one delivered via a website (Healthy Paths) and the other delivered via an app (Healthy Mind). Design modifications were introduced within Healthy Mind to take account of reported differences in how individuals engage with websites compared to apps and mobile phones. Data were collected as part of an observational study nested within a broader exploratory trial of Healthy Mind. Objective usage of Healthy Paths and Healthy Mind were automatically recorded, including frequency and duration of logins, access to specific components within the intervention and order of page/screen visits. Usage was compared for a two week period following initial registration. In total, 381 participants completed the registration process for Healthy Paths (web) and 162 participants completed the registration process for Healthy Mind (app). App users logged in twice as often (Mdn = 2.00) as web users (Mdn = 1.00), U = 13,059.50, p ≤ 0.001, but spent half as much time (Mdn = 5.23 min) on the intervention compared to web users (Mdn = 10.52 min), U = 19,740.00, p ≤ 0.001. Visual exploration of usage patterns over time revealed that a significantly higher proportion of app users (n = 126, 82.35%) accessed both types of support available within the intervention (i.e. awareness and change-focused tools) compared to web users (n = 92, 40.17%), χ2(1, n = 382) = 66.60, p < 0.001. This study suggests that the digital platform used to deliver an intervention (i.e. web versus app) and specific design choices (e.g. navigation, length and volume of content) may be associated with differences in how the intervention content is used. Broad summative usage data (e.g. total time spent on the intervention) may mask important differences in how an intervention is used by different user groups if it is not complemented by more fine-grained analyses of usage patterns over time. Trial registration number: ISRCTN67177737.

Engagement with a Web-Based Health Promotion Intervention among Vocational School Students: A Secondary User and Usage Analysis

“…First, the selected usage parameter, i.e., logins, gives a broad insight into usage frequency but not into deeper engagement [92,93]. Usage or engagement can be distinguished between extent of usage as behavior, i.e., amount or frequency, and subjective experience, e.g., attention and interest [35].…”

A Framework for Analyzing and Measuring Usage and Engagement Data (AMUsED) in Digital Interventions: Viewpoint (Preprint)

“…movement through pages) requires alternative methods to identify and inform subsequent statistical analysis. Therefore, techniques such as visualisation and process mining may be more informative as they can reveal patterns of usage within the data, such as workflow through an intervention, clustering by participant groups, and temporal details [10,[21][22][23][24]. In applying stage 3 of the framework, the required data format for any analytical software should be considered.…”

“…how an intervention worked, when and for whom [6,7]. Recent interest in 'effective engagement' considers these individual patterns of usage and the minimum level of engagement necessary for changes in target behavior to occur, including variation across individuals [6,[8][9][10]. Effective engagement may be achieved through using the intervention a certain amount.…”

“…Effective engagement may be achieved through using the intervention a certain amount. Alternatively it may be context-dependent whereby viewing specific content, in a certain order, or at an appropriate time, provides the minimum threshold necessary for change [6,9,10].…”

“…Researchers have recently advocated using DBCI data in this way [9,10,18]. However, efficient management and systematic analysis of large data sets can be challenging in the absence of detailed guiding frameworks [8], particularly if specific testable hypotheses/questions about usage have not been considered in advance of data collection.…”