Toward a Mobile Platform for Real-world Digital Measurement of Depression: User-Centered Design, Data Quality, and Behavioral and Clinical Modeling

Abstract

Background Although effective mental health treatments exist, the ability to match individuals to optimal treatments is poor, and timely assessment of response is difficult. One reason for these challenges is the lack of objective measurement of psychiatric symptoms. Sensors and active tasks recorded by smartphones provide a low-burden, low-cost, and scalable way to capture real-world data from patients that could augment clinical decision-making and move the field of mental health closer to measurement-based care. Objective This study tests the feasibility of a fully remote study on individuals with self-reported depression using an Android-based smartphone app to collect subjective and objective measures associated with depression severity. The goals of this pilot study are to develop an engaging user interface for high task adherence through user-centered design; test the quality of collected data from passive sensors; start building clinically relevant behavioral measures (features) from passive sensors and active inputs; and preliminarily explore connections between these features and depression severity. Methods A total of 600 participants were asked to download the study app to join this fully remote, observational 12-week study. The app passively collected 20 sensor data streams (eg, ambient audio level, location, and inertial measurement units), and participants were asked to complete daily survey tasks, weekly voice diaries, and the clinically validated Patient Health Questionnaire (PHQ-9) self-survey. Pairwise correlations between derived behavioral features (eg, weekly minutes spent at home) and PHQ-9 were computed. Using these behavioral features, we also constructed an elastic net penalized multivariate logistic regression model predicting depressed versus nondepressed PHQ-9 scores (ie, dichotomized PHQ-9). Results A total of 415 individuals logged into the app. Over the course of the 12-week study, these participants completed 83.35% (4151/4980) of the PHQ-9s. Applying data sufficiency rules for minimally necessary daily and weekly data resulted in 3779 participant-weeks of data across 384 participants. Using a subset of 34 behavioral features, we found that 11 features showed a significant (P<.001 Benjamini-Hochberg adjusted) Spearman correlation with weekly PHQ-9, including voice diary–derived word sentiment and ambient audio levels. Restricting the data to those cases in which all 34 behavioral features were present, we had available 1013 participant-weeks from 186 participants. The logistic regression model predicting depression status resulted in a 10-fold cross-validated mean area under the curve of 0.656 (SD 0.079). Conclusions This study finds a strong proof of concept for the use of a smartphone-based assessment of depression outcomes. Behavioral features derived from passive sensors and active tasks show promising correlations with a validated clinical measure of depression (PHQ-9). Future work is needed to increase scale that may permit the construction of more complex (eg, nonlinear) predictive models and better handle data missingness.

Digital phenotyping correlations in larger mental health samples: analysis and replication

“…In another large study of 288 participants studying mood and anxiety, Meyerhoff et al 6 employed a different approach, looking at correlations between changes in weekly survey scores and changes in passive data features. Focusing on GPS, call, text and app usage features, this study also reported low correlations similar to Nickels et al 5 Meyerhoff et al also separated participants into groups, using k -means clustering on the participants’ clinical scores, and found that some correlations were higher in groups exhibiting symptoms. 6 In this work, we aim to explore correlations in a large data-set collected with the mindLAMP app from college student participants, to assess if we observe correlations of a similar magnitude to Nickels et al 5 and Meyerhoff et al 6 In addition, we explore whether changing the group of participants that we use for analysis (such as by setting data-quality thresholds or by splitting into clinical groups) will allow us to identify more clinically meaningful correlations.…”

“…Focusing on GPS, call, text and app usage features, this study also reported low correlations similar to Nickels et al 5 Meyerhoff et al also separated participants into groups, using k -means clustering on the participants’ clinical scores, and found that some correlations were higher in groups exhibiting symptoms. 6 In this work, we aim to explore correlations in a large data-set collected with the mindLAMP app from college student participants, to assess if we observe correlations of a similar magnitude to Nickels et al 5 and Meyerhoff et al 6 In addition, we explore whether changing the group of participants that we use for analysis (such as by setting data-quality thresholds or by splitting into clinical groups) will allow us to identify more clinically meaningful correlations. Finally, we aim to test a classifier for predicting survey scores with passive and survey data, to assess whether passive data signals alone are enough to build predictive models or if survey data is necessary to provide a stronger signal.…”

“…We hope that future work will continue to test the validity of passive data in larger and more diverse samples. Unlike in related studies, 5,6 since mindLAMP is an open-source platform, researchers can reproduce our study. The LAMP consortium currently consists of over 50 sites around the world, encouraging result sharing and comparisons across different populations.…”

“…That said, it is possible that digital phenotyping clinical correlates may not reflect current clinical concepts, but such a data-driven approach will require larger-scale studies than any mentioned in this paper. Like other studies that have struggled to replicate digital phenotyping results, 17 we also did not capture the exact same data streams, derive the exact same features or use the exact same types of smartphones as in either Meyerhoff et al 6 or Nickels et al 5 These sources of natural variation present challenges to any replication efforts, and as we showed in Figure 1(a) and 1(b), there are some differences if participants with poorer data quality are included in the analysis.…”

Similarity matrix-based anomaly detection for clinical intervention

“…Our results are in line with those derived from more traditional methods. For example a recent paper from Verily 10 found that smartphone mobility metrics were also correlated with mood. Contrarily, another paper 11 with a sample of 255 participants found that there were no significant associations between changes in symptom severity measures and subsequent changes in sensor-derived behavioral features.…”