Toward Generalizable and Transdiagnostic Tools for Psychosis Prediction: An Independent Validation and Improvement of the NAPLS-2 Risk Calculator in the Multisite PRONIA Cohort

Koutsouleris, Nikolaos; Worthington, Michelle; Dwyer, Dominic; Kambeitz-Ilankovic, Lana; Sanfelici, Rachele; Fusar‐Poli, Paolo; Rosen, Marlene; Ruhrmann, Stephan; Antičević, Alan; Addington, Jean; Perkins, Diana O.; Bearden, Carrie E.; Cornblatt, Barbara A.; Cadenhead, Kristin S.; Mathalon, Daniel H.; McGlashan, Thomas H.; Seidman, Larry J.; Tsuang, Ming T.; Walker, Elaine F.; Woods, Scott W.; Falkai, Peter; Lencer, Rebekka; Bertolino, Alessandro; Kambeitz, Joseph; Schultze‐Lutter, Frauke; Meisenzahl, Eva; Salokangas, Raimo K. R.; Hietala, Jarmo; Brambilla, Paolo; Upthegrove, Rachel; Borgwardt, Stefan; Wood, Stephen; Gur, Raquel E.; McGuire, Philip; Cannon, Tyrone D.

doi:10.1016/j.biopsych.2021.06.023

Cited by 34 publications

(20 citation statements)

References 75 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thus, the prediction of such functional outcomes should be targeting both CHRs and affective samples, to obtain more realistic, reliable and potentially transdiagnostic prediction models of future risk for further functional impairments and, ultimately, disability, in a more heterogeneous help-seeking population. 7,25,26 Employing machine learning to identify reliable functioning predictors So far, research on early identification of patients who subsequently develop psychosis or other adverse outcomes has produced favourable results, yet further improvement is required. Few studies 27,28 have tested the predictive value of environmental adverse events on functional impairments across concurrent psychiatric conditions, as frequently present in psychosis risk syndromes.…”

Section: Discussionmentioning

confidence: 99%

“…Permutation-based pairwise comparisons between discovery unimodal and multimodal classifier performance are reported in Supplementary Table 5. As a check, we have calculated the expected calibration error (ECE) 25,53 to estimate calibration for the models achieving best accuracy and generalisability in our discovery cohorts (see Results). The ECE was relatively low, although not perfect (mean ECE = 0.21).…”

Section: Methodsmentioning

confidence: 99%

Section: Functioning Impairments and Risk For Psychosis And Depressionmentioning

confidence: 99%

“…52 We purposefully did not investigate the joint predictive ability of clinical and sMRI classifiers, as this was already explicitly addressed in a recent publication from our group. 7 To measure the discriminative utility of the input variables within each unimodal classifier, we computed the probability of being selected for classification purposes within the inner cross-validation loop for each feature, 27 following a 25,53 to estimate calibration for the models achieving best accuracy and generalisability in our discovery cohorts (see Results). The ECE was relatively low, although not perfect (mean ECE = 0.21).…”

Section: Machine Learning Pipelinementioning

confidence: 99%

See 3 more Smart Citations

Using combined environmental–clinical classification models to predict role functioning outcome in clinical high-risk states for psychosis and recent-onset depression

et al. 2022

Self Cite

View full text Add to dashboard Cite

Background Clinical high-risk states for psychosis (CHR) are associated with functional impairments and depressive disorders. A previous PRONIA study predicted social functioning in CHR and recent-onset depression (ROD) based on structural magnetic resonance imaging (sMRI) and clinical data. However, the combination of these domains did not lead to accurate role functioning prediction, calling for the investigation of additional risk dimensions. Role functioning may be more strongly associated with environmental adverse events than social functioning. Aims We aimed to predict role functioning in CHR, ROD and transdiagnostically, by adding environmental adverse events-related variables to clinical and sMRI data domains within the PRONIA sample. Method Baseline clinical, environmental and sMRI data collected in 92 CHR and 95 ROD samples were trained to predict lower versus higher follow-up role functioning, using support vector classification and mixed k-fold/leave-site-out cross-validation. We built separate predictions for each domain, created multimodal predictions and validated them in independent cohorts (74 CHR, 66 ROD). Results Models combining clinical and environmental data predicted role outcome in discovery and replication samples of CHR (balanced accuracies: 65.4% and 67.7%, respectively), ROD (balanced accuracies: 58.9% and 62.5%, respectively), and transdiagnostically (balanced accuracies: 62.4% and 68.2%, respectively). The most reliable environmental features for role outcome prediction were adult environmental adjustment, childhood trauma in CHR and childhood environmental adjustment in ROD. Conclusions Findings support the hypothesis that environmental variables inform role outcome prediction, highlight the existence of both transdiagnostic and syndrome-specific predictive environmental adverse events, and emphasise the importance of implementing real-world models by measuring multiple risk dimensions.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Functioning Impairments and Risk For Psychosis And Depressionmentioning

confidence: 99%

Section: Machine Learning Pipelinementioning

confidence: 99%

See 2 more Smart Citations

Using combined environmental–clinical classification models to predict role functioning outcome in clinical high-risk states for psychosis and recent-onset depression

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…2 Using algorithmic pattern recognition, this work showed better accuracy than human prognosis. The North American Prodrome Longitudinal Study (NAPLS) individual risk calculator for development of psychosis from the clinical high-risk (CHR) state 3 has been recently validated in more broadly defined CHR groups from multiple countries, 4 including patients with recent-onset depression from the PRONIA consortia. This valuable generalisable model points to younger age at onset and reduced cognitive processing speed as increasingly relevant risk markers in broader risk cohorts.…”

Section: Prognostic Riskmentioning

confidence: 99%

The potential of precision psychiatry: what is in reach?

2022

Self Cite

View full text Add to dashboard Cite

SummaryProgress in developing personalised care for mental disorders is supported by numerous proof-of-concept machine learning studies in the area of risk assessment, diagnostics and precision prescribing. Most of these studies primarily use clinical data, but models might benefit from additional neuroimaging, blood and genetic data to improve accuracy. Combined, multimodal models might offer potential for stratification of patients for treatment. Clinical implementation of machine learning is impeded by a lack of wider generalisability, with efforts primarily focused on psychosis and dementia. Studies across all diagnostic groups should work to test the robustness of machine learning models, which is an essential first step to clinical implementation, and then move to prospective clinical validation. Models need to exceed clinicians’ heuristics to be useful, and safe, in routine decision-making. Engagement of clinicians, researchers and patients in digitalisation and ‘big data’ approaches are vital to allow the generation and accessibility of large, longitudinal, prospective data needed for precision psychiatry to be applied into real-world psychiatric care.

show abstract

Recent Updates on Predicting Conversion in Youth at Clinical High Risk for Psychosis

Caballero,

Machiraju,

Diomino

et al. 2023

Curr Psychiatry Rep

View full text Add to dashboard Cite

Purpose of Review This review highlights recent advances in the prediction and treatment of psychotic conversion. Over the past 25 years, research into the prodromal phase of psychotic illness has expanded with the promise of early identification of individuals at clinical high risk (CHR) for psychosis who are likely to convert to psychosis. Recent Findings Meta-analyses highlight conversion rates between 20 and 30% within 2–3 years using existing clinical criteria while research into more specific risk factors, biomarkers, and refinement of psychosis risk calculators has exploded, improving our ability to predict psychotic conversion with greater accuracy. Recent studies highlight risk factors and biomarkers likely to contribute to earlier identification and provide insight into neurodevelopmental abnormalities, CHR subtypes, and interventions that can target specific risk profiles linked to neural mechanisms. Summary Ongoing initiatives that assess longer-term (> 5–10 years) outcome of CHR participants can provide valuable information about predictors of later conversion and diagnostic outcomes while large-scale international biomarker studies provide hope for precision intervention that will alter the course of early psychosis globally.

show abstract

Toward Generalizable and Transdiagnostic Tools for Psychosis Prediction: An Independent Validation and Improvement of the NAPLS-2 Risk Calculator in the Multisite PRONIA Cohort

Cited by 34 publications

References 75 publications

Using combined environmental–clinical classification models to predict role functioning outcome in clinical high-risk states for psychosis and recent-onset depression

Using combined environmental–clinical classification models to predict role functioning outcome in clinical high-risk states for psychosis and recent-onset depression

The potential of precision psychiatry: what is in reach?

Recent Updates on Predicting Conversion in Youth at Clinical High Risk for Psychosis

Contact Info

Product

Resources

About