Using machine learning with intensive longitudinal data to predict depression and suicidal ideation among medical interns over time

Abstract

Background Use of intensive longitudinal methods (e.g. ecological momentary assessment, passive sensing) and machine learning (ML) models to predict risk for depression and suicide has increased in recent years. However, these studies often vary considerably in length, ML methods used, and sources of data. The present study examined predictive accuracy for depression and suicidal ideation (SI) as a function of time, comparing different combinations of ML methods and data sources. Methods Participants were 2459 first-year training physicians (55.1% female; 52.5% White) who were provided with Fitbit wearable devices and assessed daily for mood. Linear [elastic net regression (ENR)] and non-linear (random forest) ML algorithms were used to predict depression and SI at the first-quarter follow-up assessment, using two sets of variables (daily mood features only, daily mood features + passive-sensing features). To assess accuracy over time, models were estimated iteratively for each of the first 92 days of internship, using data available up to that point in time. Results ENRs using only the daily mood features generally had the best accuracy for predicting mental health outcomes, and predictive accuracy within 1 standard error of the full 92 day models was attained by weeks 7–8. Depression at 92 days could be predicted accurately (area under the curve >0.70) after only 14 days of data collection. Conclusions Simpler ML methods may outperform more complex methods until passive-sensing features become better specified. For intensive longitudinal studies, there may be limited predictive value in collecting data for more than 2 months.

Passive Sensing for the Prediction of Suicidal Thoughts and Behaviors: A Systematic Review and Recommendations for Future Research

“…1 for the study selection process). A total of 75 full texts were screened, revealing eight studies investigating the predictive value of passive sensing for the prediction of STB [25][26][27][28][29][30][31][32], six trials focusing on the feasibility of passive sensing [27,[33][34][35][36][37], and ve study protocols [38][39][40][41][42]. All papers were published between 2019 and 2022.…”

“…One study recruited adolescents [32], whereas the other studies were in adult populations. [25,32], and four studies used both [29][30][31]. The passive data collection interval varied between seven [28, 32] and 92 days [29,30].…”

“…[25,32], and four studies used both [29][30][31]. The passive data collection interval varied between seven [28, 32] and 92 days [29,30]. Two studies provided a detailed listing of the device sensors and analytics, as well as sampling frequency [31,32].…”

“…Studies differed in how they assessed STB. Several studies assessed suicidal ideation using single items of validated scales, (e.g., item 9 from the PHQ-9) [25,27,29,30]. Furthermore, the Columbia-Suicide Severity Rating Scale (C-SSRS) was frequently used [25,28,32], and one study used the Suicide Behaviors Questionnaire (SBQ) [26].…”

“…The studies using the C-SSRS and the SBQ used a composite score, which did not differentiate between suicidal thoughts and behaviors. Studies divided the outcome into the presence or absence of STB (6/8 studies) [27][28][29][30][31][32] and/or evaluated STB on a severity level (4/8 studies) [25,26,31,32]. Seven studies assessed STB between one and three times in total, and one study used three EMA items to collect momentary data on suicidal ideation several times a day [31].…”