Assessment of Real-World Use of Behavioral Health Mobile Applications by a Novel Stickiness Metric

Carlo, Andrew D.; Ghomi, Reza Hosseini; Renn, Brenna N.; Strong, Michael A.; Areán, Patricia A.

doi:10.1001/jamanetworkopen.2020.11978

Cited by 26 publications

(22 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Because retention poses a significant challenge for entirely unguided treatment programs, our finding that 60.44% (2730/4517) of users continued to engage with the app in the second week and 42.66% (1927/4517) of users continued to engage with the app in the fourth week after initial download is promising. Although there are no clearly established metrics of retention for mobile apps, a recent paper examining retention among different mobile apps showed that Youper had the highest “stickiness” (measured by the ratio of active users to downloads in a given month) compared to any other treatment app for anxiety and depression [ 57 ]. Because Youper users experienced symptom improvements on average within the first 2 weeks of app use, with the present retention rate, it is likely that a large portion of users will stick with the app long enough to experience some positive effects.…”

Section: Discussionmentioning

confidence: 99%

Acceptability and Effectiveness of Artificial Intelligence Therapy for Anxiety and Depression (Youper): Longitudinal Observational Study

Mehta¹,

Niles²,

Vargas³

et al. 2021

J Med Internet Res

View full text Add to dashboard Cite

Background Youper is a widely used, commercially available mobile app that uses artificial intelligence therapy for the treatment of anxiety and depression. Objective Our study examined the acceptability and effectiveness of Youper. Further, we tested the cumulative regulation hypothesis, which posits that cumulative emotion regulation successes with repeated intervention engagement will predict longer-term anxiety and depression symptom reduction. Methods We examined data from paying Youper users (N=4517) who allowed their data to be used for research. To characterize the acceptability of Youper, we asked users to rate the app on a 5-star scale and measured retention statistics for users’ first 4 weeks of subscription. To examine effectiveness, we examined longitudinal measures of anxiety and depression symptoms. To test the cumulative regulation hypothesis, we used the proportion of successful emotion regulation attempts to predict symptom reduction. Results Youper users rated the app highly (mean 4.36 stars, SD 0.84), and 42.66% (1927/4517) of users were retained by week 4. Symptoms decreased in the first 2 weeks of app use (anxiety: d=0.57; depression: d=0.46). Anxiety improvements were maintained in the subsequent 2 weeks, but depression symptoms increased slightly with a very small effect size (d=0.05). A higher proportion of successful emotion regulation attempts significantly predicted greater anxiety and depression symptom reduction. Conclusions Youper is a low-cost, completely self-guided treatment that is accessible to users who may not otherwise access mental health care. Our findings demonstrate the acceptability and effectiveness of Youper as a treatment for anxiety and depression symptoms and support continued study of Youper in a randomized clinical trial.

show abstract

Section: Discussionmentioning

confidence: 99%

Acceptability and Effectiveness of Artificial Intelligence Therapy for Anxiety and Depression (Youper): Longitudinal Observational Study

Mehta¹,

Niles²,

Vargas³

et al. 2021

J Med Internet Res

View full text Add to dashboard Cite

show abstract

“…Importantly, an app’s MAU reflects two distinct components: (1) the number of people who downloaded the app (which reflects an app’s popularity and marketing success) and (2) retention (which may reflect content and features within the app). Because retention data are often difficult to obtain, investigators recently proposed an alternative “stickiness” metric, defined as the number of monthly active users per normalized total downloads [ 12 ]. Interestingly, some of the most downloaded apps do not appear to be particularly sticky, and some of the stickiest apps are not the most downloaded [ 12 ].…”

Section: Introductionmentioning

confidence: 99%

“…Because retention data are often difficult to obtain, investigators recently proposed an alternative “stickiness” metric, defined as the number of monthly active users per normalized total downloads [ 12 ]. Interestingly, some of the most downloaded apps do not appear to be particularly sticky, and some of the stickiest apps are not the most downloaded [ 12 ].…”

Section: Introductionmentioning

confidence: 99%

User Experience, Engagement, and Popularity in Mental Health Apps: Secondary Analysis of App Analytics and Expert App Reviews

Kaveladze¹,

Wasil²,

Bunyi³

et al. 2022

JMIR Hum Factors

View full text Add to dashboard Cite

Background User experience and engagement are critical elements of mental health apps’ abilities to support users. However, work examining the relationships among user experience, engagement, and popularity has been limited. Understanding how user experience relates to engagement with and popularity of mental health apps can demonstrate the relationship between subjective and objective measures of app use. In turn, this may inform efforts to develop more effective and appealing mental health apps and ensure that they reach wide audiences. Objective We aimed to examine the relationship among subjective measures of user experience, objective measures of popularity, and engagement in mental health apps. Methods We conducted a preregistered secondary data analysis in a sample of 56 mental health apps. To measure user experience, we used expert ratings on the Mobile App Rating Scale (MARS) and consumer ratings from the Apple App Store and Google Play. To measure engagement, we acquired estimates of monthly active users (MAU) and user retention. To measure app popularity, we used download count, total app revenue, and MAU again. Results MARS total score was moderately positively correlated with app-level revenue (Kendall rank [T]=0.30, P=.002), MAU (T=0.39, P<.001), and downloads (T=0.41, P<.001). However, the MARS total score and each of its subscales (Engagement, Functionality, Aesthetics, and Information) showed extremely small correlations with user retention 1, 7, and 30 days after downloading. Furthermore, the total MARS score only correlated with app store rating at T=0.12, which, at P=.20, did not meet our threshold for significance. Conclusions More popular mental health apps receive better ratings of user experience than less popular ones. However, user experience does not predict sustained engagement with mental health apps. Thus, mental health app developers and evaluators need to better understand user experience and engagement, as well as to define sustained engagement, what leads to it, and how to create products that achieve it. This understanding might be supported by better collaboration between industry and academic teams to advance a science of engagement.

show abstract

“…First, we did not have access to participant-level demographic information such as race or socioeconomic status, which may be important determinants of engagement and medication adherence. Second, while we measured two objective measures of engagement, other objective measures like app push notifications and reminders, daily and monthly active user ratios (DAU/ MAU) 29 , number of in-app clicks, opening of app content pages, etc. may also provide useful signals.…”

Section: Discussionmentioning

confidence: 99%

The relationship between objective app engagement and medication adherence in asthma and COPD: a retrospective analysis

Kaye

Gondalia

Thompson

et al. 2021

Sci Rep

View full text Add to dashboard Cite

Digital health tools can promote disease self-management, but the association of smartphone app engagement and medication adherence is unclear. We assessed the relationship between objective smartphone app engagement and controller medication use in adults with asthma and COPD. We retrospectively analyzed data from participants enrolled in a digital self-management platform for asthma and COPD. Eligible adults had a smartphone and a paired electronic medication monitor (EMM). Longitudinal, mixed-effects logistic regressions estimated the relationship between daily app engagement (app opens, session duration) and daily controller medication use. Data from 2309 participants (71% asthma; 29% COPD) was analyzed. Opening the app (vs. not opening the app) was associated with significantly greater odds (OR (95% CI)) of using controller medications in asthma (2.08 (1.98, 2.19)) and COPD (1.61 (1.49, 1.75). Longer session duration was also associated with greater odds of using controller medications in asthma and COPD, but the odds of use attenuated with longer session duration in COPD. This study presents a novel assessment of the relationship between objectively-measured smartphone app engagement and controller medication use in asthma and COPD. Such insights may help develop targeted digital health tools and interventions.

show abstract

Assessment of Real-World Use of Behavioral Health Mobile Applications by a Novel Stickiness Metric

Abstract: This cross-sectional study uses a novel “stickiness” measure to evaluate which mobile applications for smoking cessation, meditation, and other behavioral health areas are associated with long-term engagement.

Cited by 26 publications

References 6 publications

Acceptability and Effectiveness of Artificial Intelligence Therapy for Anxiety and Depression (Youper): Longitudinal Observational Study

Acceptability and Effectiveness of Artificial Intelligence Therapy for Anxiety and Depression (Youper): Longitudinal Observational Study

User Experience, Engagement, and Popularity in Mental Health Apps: Secondary Analysis of App Analytics and Expert App Reviews

The relationship between objective app engagement and medication adherence in asthma and COPD: a retrospective analysis

Contact Info

Product

Resources

About