Predicting remission after internet-delivered psychotherapy in patients with depression using machine learning and multi-modal data

Abstract

This study applied supervised machine learning with multi-modal data to predict remission of major depressive disorder (MDD) after psychotherapy. Genotyped adult patients (n = 894, 65.5% women, age 18–75 years) diagnosed with mild-to-moderate MDD and treated with guided Internet-based Cognitive Behaviour Therapy (ICBT) at the Internet Psychiatry Clinic in Stockholm were included (2008–2016). Predictor types were demographic, clinical, process (e.g., time to complete online questionnaires), and genetic (polygenic risk scores). Outcome was remission status post ICBT (cut-off ≤10 on MADRS-S). Data were split into train (60%) and validation (40%) given ICBT start date. Predictor selection employed human expertise followed by recursive feature elimination. Model derivation was internally validated through cross-validation. The final random forest model was externally validated against a (i) null, (ii) logit, (iii) XGBoost, and (iv) blended meta-ensemble model on the hold-out validation set. Feature selection retained 45 predictors representing all four predictor types. With unseen validation data, the final random forest model proved reasonably accurate at classifying post ICBT remission (Accuracy 0.656 [0.604, 0.705], P vs null model = 0.004; AUC 0.687 [0.631, 0.743]), slightly better vs logit (bootstrap D = 1.730, P = 0.084) but not vs XGBoost (D = 0.463, P = 0.643). Transparency analysis showed model usage of all predictor types at both the group and individual patient level. A new, multi-modal classifier for predicting MDD remission status after ICBT treatment in routine psychiatric care was derived and empirically validated. The multi-modal approach to predicting remission may inform tailored treatment, and deserves further investigation to attain clinical usefulness.

Predicting outcomes at the individual patient level: what is the best method?

“…However, we deemed this highly unlikely as we previously compared neural networks with ridge regression for the same data set and found almost identical predictive performance 18. Second, our analysis focused only on pharmacological treatments for depression rather than other non-pharmacological treatments such as psychotherapy, which may limit the generalisability of findings as the prediction of non-pharmacological treatment outcomes might require a completely different set of predictors and thus some strong predictors might be found 28. Finally, we did not consider neuroimaging and genomic multimodal data in our analyses, which could potentially introduce non-linearities, enlarge the separation between clusters and benefit the predictive performance of the similarity-based approach 29 30.…”

“… 18 Second, our analysis focused only on pharmacological treatments for depression rather than other non-pharmacological treatments such as psychotherapy, which may limit the generalisability of findings as the prediction of non-pharmacological treatment outcomes might require a completely different set of predictors and thus some strong predictors might be found. 28 Finally, we did not consider neuroimaging and genomic multimodal data in our analyses, which could potentially introduce non-linearities, enlarge the separation between clusters and benefit the predictive performance of the similarity-based approach. 29 30 The performance of complex models such as deep neural networks should be further assessed in future studies using variables from different international data sets and across other mental health disorders and treatments.…”

Clinical, genetic, and sociodemographic predictors of symptom severity after internet-delivered cognitive behavioural therapy for depression and anxiety

“…Preliminary evidence suggests a link between a higher polygenic loading for autism spectrum disorder (PRS ASD) and a poorer treatment response in patients undergoing ICBT for MDD 41 . In another study using the same data as 41 with machine learning modelling, higher PRS MDD and PRS for intelligence (PRS IQ) weakly predicted remission in ICBT for MDD 42 . However, the largest therapygenetic GWA meta-analysis to date (n = 2724) did not identify any common genetic variants associated with the CBT treatment outcome for depression and anxiety 43 , consistent with a previous GWAS that had a similar sample size and was also likely underpowered 44 .…”

Swedish multimodal cohort of patients with anxiety or depression treated with internet-delivered psychotherapy (MULTI-PSYCH)

“…Another study used the same sample applying supervised machine learning for prediction of MDD remission, with a particular focus on the potential utility of multimodal (clinical, genetic and register) predictors available at baseline before treatment with ICBT. The study derived a new, multimodal classifier for prediction of MDD remission status 76. In terms of feature engineering, a study applying longitudinal clustering of weekly symptom trajectories consisting of weekly symptom ratings will investigate if a data-driven clustering of symptoms across time during ICBT can be predictive of clinical and socioeconomic outcomes, the latter in linked registers 77.…”