Predicting one-year outcome in first episode psychosis using machine learning

Abstract

Background Early illness course correlates with long-term outcome in psychosis. Accurate prediction could allow more focused intervention. Earlier intervention corresponds to significantly better symptomatic and functional outcomes. Our study objective is to use routinely collected baseline demographic and clinical characteristics to predict employment, education or training (EET) status, and symptom remission in patients with first episode psychosis (FEP) at one-year. Methods and findings 83 FEP patients were recruited from National Health Service (NHS) Glasgow between 2011 and 2014 to a 24-month prospective cohort study with regular assessment of demographic and psychometric measures. An external independent cohort of 79 FEP patients were recruited from NHS Glasgow and Edinburgh during a 12-month study between 2006 and 2009. Elastic net regularised logistic regression models were built to predict binary EET status, period and point remission outcomes at one-year on 83 Glasgow patients (training dataset). Models were externally validated on an independent dataset of 79 patients from Glasgow and Edinburgh (validation dataset). Only baseline predictors shared across both cohorts were made available for model training and validation. After excluding participants with missing outcomes, models were built on the training dataset for EET status, period and point remission outcomes and externally validated on the validation dataset. Models predicted EET status, period and point remission with receiver operating curve (ROC) area under the curve (AUC) performances of 0.876 (95%CI: 0.864, 0.887), 0.630 (95%CI: 0.612, 0.647) and 0.652 (95%CI: 0.635, 0.670) respectively. Positive predictors of EET included baseline EET and living with spouse/children. Negative predictors included higher PANSS suspiciousness, hostility and delusions scores. Positive predictors for symptom remission included living with spouse/children, and affective symptoms on the Positive and Negative Syndrome Scale (PANSS). Negative predictors of remission included passive social withdrawal symptoms on PANSS. A key limitation of this study is the small sample size (n) relative to the number of predictors (p), whereby p approaches n. The use of elastic net regularised regression rather than ordinary least squares regression helped circumvent this difficulty. Further, we did not have information for biological and additional social variables, such as nicotine dependence, which observational studies have linked to outcomes in psychosis. Conclusions and relevance Using advanced statistical machine learning techniques, we provide the first externally validated evidence, in a temporally and geographically independent cohort, for the ability to predict one-year EET status and symptom remission in individual FEP patients.

Real-world digital implementation of the Psychosis Polyrisk Score (PPS): A pilot feasibility study

“…The PPS biomarker, if integrated into Electronic Health Records or website approaches (such as YouR-Study) could then become the first entry point in a stepped risk stratification framework. More labour- and time-intensive biomarkers, such as those that rely on cognitive assessments ( Cannon et al, 2016 ), clinical assessments ( Cannon et al, 2016 ; Leighton et al, 2019a ; Leighton et al, 2019b ) or neuroimaging ( Koutsouleris et al, 2009 ; Koutsouleris et al, 2012a , Koutsouleris et al, 2015 ; Koutsouleris et al, 2018 ) could be reserved to those individuals initially detected through PPS screening ( Fusar-Poli et al, 2019b ). However, more work needs to be done to assess the prognostic ability of the PPS.…”

Predicting One-Year Outcome in First Episode Psychosis using Machine Learning

“…Few such relationships are externally validated and, of those that are, many lose significance. [45,46] Because of the inherent feature selection of regularised logistic regression classifiers, which avoids overfitting in ill-conditioned regression problems where the number of variables is close to the sample size, our models are sparse and uniquely interpretable. Further, without a priori selection of variables but instead allowing the classifier to select features, we avoid the introduction of additional observer bias.…”

Predicting patient outcomes in psychiatric hospitals with routine data: a machine learning approach

“…Recent studies have often used a broad range of feature variables in studies restricted to specific settings and patients. Leigthon et al [47] predicted remission after 12 months in 79 patients with first episode of psychosis with a wide range of demographic, socioeconomic and psychometric feature variables and reached an area under the ROC of 0.65. Koutsouleris et al [48] also investigated remission in first episode of psychosis and reached a sensitivity of 71%, a specificity of 72% and a precision of 93% in 108 unseen patients with their top ten demographic, socioeconomic and psychometric predictor variables.…”