Generalizability of Clinical Prediction Models in Mental Health - Real-World Validation of Machine Learning Models for Depressive Symptom Prediction

Abstract

Mental health research faces the challenge of developing machine learning models for clinical decision support. Concerns about the generalizability of such models to real-world populations due to sampling effects and disparities in available data sources are rising. We examined whether harmonized, structured collection of clinical data and stringent measures against overfitting can facilitate the generalization of machine learning models for predicting depressive symptoms across diverse real-world inpatient and outpatient samples. Despite systematic differences between samples, a sparse machine learning model trained on clinical information exhibited strong generalization across diverse real-world samples. These findings highlight the crucial role of standardized routine data collection, grounded in unified ontologies, in the development of generalizable machine learning models in mental health.One-Sentence SummaryGeneralization of sparse machine learning models trained on clinical data is possible for depressive symptom prediction.