Brain–phenotype models fail for individuals who defy sample stereotypes

Abstract

Individual differences in brain functional organization track a range of traits, symptoms and behaviours1–12. So far, work modelling linear brain–phenotype relationships has assumed that a single such relationship generalizes across all individuals, but models do not work equally well in all participants13,14. A better understanding of in whom models fail and why is crucial to revealing robust, useful and unbiased brain–phenotype relationships. To this end, here we related brain activity to phenotype using predictive models—trained and tested on independent data to ensure generalizability15—and examined model failure. We applied this data-driven approach to a range of neurocognitive measures in a new, clinically and demographically heterogeneous dataset, with the results replicated in two independent, publicly available datasets16,17. Across all three datasets, we find that models reflect not unitary cognitive constructs, but rather neurocognitive scores intertwined with sociodemographic and clinical covariates; that is, models reflect stereotypical profiles, and fail when applied to individuals who defy them. Model failure is reliable, phenotype specific and generalizable across datasets. Together, these results highlight the pitfalls of a one-size-fits-all modelling approach and the effect of biased phenotypic measures18–20 on the interpretation and utility of resulting brain–phenotype models. We present a framework to address these issues so that such models may reveal the neural circuits that underlie specific phenotypes and ultimately identify individualized neural targets for clinical intervention.

Evidence for Embracing Normative Modeling

“…Prior research supports the use of age-specific templates and ethnicity specific growth charts Dong et al (2020 ). This is a major limitation which requires additional future work and should be considered carefully when transferring the model to diverse data Benkarim et al (2022 ); Greene et al (2022 ); Li et al (2022 ). The term ‘normative model’ can be defined in other fields in a very different manner than ours Colyvan (2013 ); Baron (2004 ); Catita et al (2020 ).…”

Parsing clinical and neurobiological sources of heterogeneity in depression

“…Prior work has suggested multiple neuroimaging correlates of depression, including structural, diffusion, and resting state abnormalities in the frontal cortex, deep nuclear gray matter, and temporal and parietal regions [4][5][6] . However, these findings have suffered from poor replicability 5,7 and/or small magnitude of effect 8 . Several prominent studies, both broadly in neuroimaging 7,9 and specifically in depression 5,10 , have suggested these replicability issues could largely be due to variability among patients in their clinical symptomatology and/or neurobiology, known as "patient heterogeneity".…”

“…However, these findings have suffered from poor replicability 5,7 and/or small magnitude of effect 8 . Several prominent studies, both broadly in neuroimaging 7,9 and specifically in depression 5,10 , have suggested these replicability issues could largely be due to variability among patients in their clinical symptomatology and/or neurobiology, known as "patient heterogeneity".…”

Editorial: Insights in Alzheimer's disease and related dementias

“…These landscapes of dementia science are changing rapidly, creating novel bridges across disciplines, diverse populations, regions, scales, methods and approaches. Some of these reconfigurations are driven by animal and human research focused in multiple emerging areas such as diversity contributions to genetic traits (Dehghani et al, 2021 ); heterogeneity and variation in protein misfolding and aggregation (Frisoni et al, 2022 ); explanatory models based on excitation/inhibition synaptic activity (Babiloni et al, 2020 ); impact of multiple sources of disparities (gender, admixtures, cultural, socioeconomic) (Alladi and Hachinski, 2018 ; Parra et al, 2018 , 2021 ); development of multimodal and region-specific biomarkers (Moguilner et al, 2022 ; Parra et al, 2022 ; Maito et al, 2023 ) and initiatives (Ibanez et al, 2021 ; Parra et al, 2021 ; Duran-Aniotz et al, 2022 ); interactions between environmental stressors and physiopathological mechanisms of allostatic overload (Birba et al, 2022 ; De Felice et al, 2022 ; Migeot et al, 2022 ); and going beyond universal models toward non-stereotypical samples (Greene et al, 2022 ) and designs (Ibanez, 2022 ) in neuroscience and dementia (Alladi and Hachinski, 2018 ). Notably, many of these key matters are being covered in this special issue.…”