Different scaling of linear models and deep learning in UKBiobank brain images versus machine-learning datasets

Schulz, Marc-André; Yeo, B.T. Thomas; Vogelstein, Joshua T.; Mourao-Miranada, Janaina; Kather, Jakob Nikolas; Kording, Konrad; Richards, Blake A; Bzdok, Danilo

doi:10.1038/s41467-020-18037-z

Cited by 210 publications

(195 citation statements)

References 56 publications

Supporting

Mentioning

156

Contrasting

Order By: Relevance

“…Indeed, similar conclusions have been reached in other domains, such as medicine 48,49 . Moreover, our findings support the idea that neuroimaging has not saturated the performance of simple linear models 50 . The reasons for this are manifold, and from a modeling viewpoint, it has been argued that the added value of linear "machine learning" techniques is often small, exaggerated, and does not translate into practical advantages 51 .…”

Section: Discussionsupporting

confidence: 86%

The Hard Limits of Decoding Mental States: The Decodability of fMRI

Jabakhanji

Vigotsky

Bielefeld

et al. 2020

Preprint

View full text Add to dashboard Cite

SUMMARYHigh-profile studies claim to assess mental states across individuals using multi-voxel decoders of brain activity. The fixed, fine-grained, multi-voxel patterns in these “optimized” decoders are purportedly necessary for discriminating between, and accurately identifying, mental states. Here, we present compelling evidence that the efficacy of these decoders is overstated. Across a variety of tasks, decoder patterns were not necessary. Not only were “optimized decoders” spatially imprecise and 90% redundant, but they also performed similarly to simpler decoders, built from average brain activity. We distinguish decoder performance when used for discriminating between, in contrast to identifying, mental states, and show even when discrimination performance is strong, identification can be poor. Using similarity rules, we derived novel and intuitive discriminability metrics that capture 95% and 68% of discrimination performance within- and across-subjects, respectively. These findings demonstrate that current across-subject decoders remain inadequate for real-life decision making.

show abstract

Section: Discussionsupporting

confidence: 86%

The Hard Limits of Decoding Mental States: The Decodability of fMRI

Jabakhanji

Vigotsky

Bielefeld

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…For example, their viability to learn subtle properties of complex multiscale brain imaging data and potential to scale may be hyped 25 . Perhaps as a reaction to this inflation, recent critical commentaries unfavorably compare DL with SML approaches 26 – 28 . Yet, these commentaries are limited in a few fundamental ways, and their conclusions must be considered at specific merits as reviewed next.…”

Section: Introductionmentioning

confidence: 99%

“…This is attributable to the fact that the use of pre-engineered features deprives DL of its main advantage: representation learning from raw or minimally preprocessed input space. Among these critical studies, Schulz et al 28 compare SML and DL methods on several tasks including a combined age and gender task and separate age regression and gender classification tasks. Their study 28 focuses on using several pre-engineered features and 2D representation learning on partial data (i.e., central brain slices), whereas additionally testing 3D representation learning on whole-brain gray matter voxels, reporting comparable performance for SML and DL for all analyses.…”

Section: Introductionmentioning

confidence: 99%

“…Among these critical studies, Schulz et al 28 compare SML and DL methods on several tasks including a combined age and gender task and separate age regression and gender classification tasks. Their study 28 focuses on using several pre-engineered features and 2D representation learning on partial data (i.e., central brain slices), whereas additionally testing 3D representation learning on whole-brain gray matter voxels, reporting comparable performance for SML and DL for all analyses. Similar to the use of pre-engineered features, learning from central 2D slices conveys less information than would learning from all 2D slices, or perhaps whole-brain 3D images.…”

Section: Introductionmentioning

confidence: 99%

“…Similar to the use of pre-engineered features, learning from central 2D slices conveys less information than would learning from all 2D slices, or perhaps whole-brain 3D images. Furthermore, for 3D whole-brain learning, this study 28 attempts separate age regression and gender classification tasks on whole-brain 3D images by replicating the DL model and training pipeline in Peng et al 29 . Despite using the same DL model, training pipeline, and data set, these two studies outline contradictory results for the age regression task on whole-brain images; with Peng et al reporting a significantly higher performance for DL (as compared to SML), whereas Schulz et al report no difference in performance in this task, thus demanding further confirmation for reproducible research.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Deep learning encodes robust discriminative neuroimaging representations to outperform standard machine learning

et al. 2021

View full text Add to dashboard Cite

Recent critical commentaries unfavorably compare deep learning (DL) with standard machine learning (SML) approaches for brain imaging data analysis. However, their conclusions are often based on pre-engineered features depriving DL of its main advantage — representation learning. We conduct a large-scale systematic comparison profiled in multiple classification and regression tasks on structural MRI images and show the importance of representation learning for DL. Results show that if trained following prevalent DL practices, DL methods have the potential to scale particularly well and substantially improve compared to SML methods, while also presenting a lower asymptotic complexity in relative computational time, despite being more complex. We also demonstrate that DL embeddings span comprehensible task-specific projection spectra and that DL consistently localizes task-discriminative brain biomarkers. Our findings highlight the presence of nonlinearities in neuroimaging data that DL can exploit to generate superior task-discriminative representations for characterizing the human brain.

show abstract

Clinical Promise of Brain-Phenotype Modeling

Greene

Constable

2023

JAMA Psychiatry

View full text Add to dashboard Cite

ImportanceAssessing the link between whole-brain activity and individual differences in cognition and behavior has the potential to offer insights into psychiatric disorder etiology and change the practice of psychiatry, from diagnostic clarification to intervention. To this end, recent application of predictive modeling to link brain activity to phenotype has generated significant excitement, but clinical applications have largely not been realized. This Review explores explanations for the as yet limited practical utility of brain-phenotype modeling and proposes a path forward to fulfill this clinical potential.ObservationsClinical applications of brain-phenotype models are proposed and will require coordinated collaboration across the relatively siloed fields of psychometrics and computational neuroscience. Such interdisciplinary work will maximize the reliability and validity of modeled phenotypic measures, ensuring that resulting brain-based models are interpretable and useful. The models, in turn, may shed additional light on the neurobiological systems into which each phenotypic measure taps, permitting further phenotype refinement.Conclusions and RelevanceTogether, these observations reflect an opportunity: bridging the divide between phenotypic measure development and validation and measure end use for brain-phenotype modeling holds the promise that each may inform the other, yielding more precise and useful brain-phenotype models. Such models can in turn be used to reveal the macroscale neural bases of a given phenotype, advancing basic neuroscientific understanding and identifying circuits that can be targeted (eg, via closed-loop neurofeedback or brain stimulation) to slow, reverse, or even prevent functional impairment.

show abstract

Different scaling of linear models and deep learning in UKBiobank brain images versus machine-learning datasets

Cited by 210 publications

References 56 publications

The Hard Limits of Decoding Mental States: The Decodability of fMRI

The Hard Limits of Decoding Mental States: The Decodability of fMRI

Deep learning encodes robust discriminative neuroimaging representations to outperform standard machine learning

Clinical Promise of Brain-Phenotype Modeling

Contact Info

Product

Resources

About