Degrees of algorithmic equivalence between the brain and its DNN models

Schyns, Philippe G.; Snoek, Lukas; Daube, Christoph

doi:10.1016/j.tics.2022.09.003

Cited by 22 publications

(23 citation statements)

References 84 publications

(108 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the light of these considerations, the results of our study not only indicate a surprising degree of consistency between species so evolutionary and ecologically distant as rodents and primates, but also reinforce the need to further increase the robustness, generalizability, and biological consistency of CNNs 35,39 . For instance, in the spirit of refs.…”

Section: Discussionsupporting

confidence: 62%

“…5,12 . This approach aligns with recent strides in the field of explainable artificial intelligence 34 and cognitive sciences 35 that have highlighted how modern architectures for machine vision, despite their saturating classification accuracies on challenging benchmarks, often exploit unintelligible visual strategies 36 (e.g., features in the background) that substantially differ from those used by their biological counterpart 37,38 .…”

Section: Introductionsupporting

confidence: 60%

“…When using a convolutional neural network to model biological vision, comparing the two systems in terms of their discrimination accuracy only provides a first-order assessment of the model’s ability to capture the processing performed by the biological system. In fact, the same classification choices can in principle be supported by very different visual strategies, i.e., by very different sets of diagnostic features in the input images 35 . For instance, recent studies have shown that deep neural networks, despite their high classification accuracies, often employ image processing strategies that are substantially different from those used by humans 36 .…”

Section: Resultsmentioning

confidence: 99%

See 2 more Smart Citations

Unraveling the complexity of rat object vision requires a full convolutional network - and beyond

Muratore,

Alemi,

Zoccolan

2024

Preprint

View full text Add to dashboard Cite

Despite their prominence as model systems to dissect visual cortical circuitry, it remains unclear whether rodents are capable of truly advanced processing of visual information. Here, we considered several psychophysical studies of rat object vision, and we used a deep convolutional neural network (CNN) to measure the computational complexity required to account for the patterns of rat performances reported in these studies, as well as for the animals' perceptual strategies. We found that at least half of the CNN depth was required to match the modulation of rat classification accuracy in tasks where objects underwent variations of size, position and orientation. However, the full network was needed to equal the tolerance of rat perception to more severe image manipulations, such as partial occlusion and reduction of objects to their outlines. Finally, rats displayed a perceptual strategy that was way more invariant than that of the CNN, as they more consistently relied on the same set of diagnostic features across object transformations. Overall, these results reveal an unexpected level of sophistication of rat object vision, while reinforcing the intuition that, despite their proficiency in solving challenging image classification tasks, CNNs learn solutions that only marginally match those of biological visual systems.

show abstract

Section: Discussionsupporting

confidence: 62%

Section: Introductionsupporting

confidence: 60%

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Unraveling the complexity of rat object vision requires a full convolutional network - and beyond

Muratore,

Alemi,

Zoccolan

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Therefore, the central problem of AI and computational cognitive neuroimaging of aligning the human brain to its models, implies that both should process the same features with the same algorithmic computations to yield similar behaviors 71,72 . By adopting a componential modelling approach, we facilitate a shift from treating the brain and our models as opaque ("black boxes") to more transparent "glass boxes."…”

Section: Discussionmentioning

confidence: 99%

The Brain Computes Dynamic Facial Movements for Emotion Categorization Using a Third Pathway

Yan,

Zhan,

Garrod

et al. 2024

Preprint

Self Cite

View full text Add to dashboard Cite

Recent theories suggest a new brain pathway dedicated to processing social movement is involved in understanding emotions from biological motion, beyond the well-known ventral and dorsal pathways. However, how this social pathway functions as a network that computes dynamic biological motion signals for perceptual behavior is unchartered. Here, we used a generative model of important facial movements that participants (N = 10) categorized as "happy," "surprise," "fear," "anger," "disgust," "sad" while we recorded their MEG brain responses. Using new representational interaction measures (between facial features, MEGt source, and behavioral responses), we reveal per participant a functional social pathway extending from occipital cortex to superior temporal gyrus. Its MEG sources selectively represent, communicate and compose facial movements to disambiguate emotion categorization behavior, while occipital cortex swiftly filters out task-irrelevant identity-defining face shape features. Our findings reveal how social pathway selectively computes complex dynamic social signals to categorize emotions in individual participants.

show abstract

“…Finally, comparison of brain representations with computational models in general also has certain limits (Doerig, Sommers, et al, 2022). Recent advances in developing more interpretable (Schyns et al, 2022; Soulos & Isik, 2020) and ecological models (Kietzmann et al, 2019; Mehrer et al, 2021) of human cognition will be helpful to address them.…”

Section: Discussionmentioning

confidence: 99%

Neural computations in prosopagnosia

Faghel-Soubeyrand

Richoz

Waeber

et al. 2022

Preprint

View full text Add to dashboard Cite

We aimed to identify neural computations underlying the loss of face identification ability by modelling the brain activity of brain-lesioned patient PS, a well-documented case of acquired pure prosopagnosia. We collected a large dataset of high-density electrophysiological (EEG) recordings from PS and neurotypicals while they completed a one-back task on a stream of face, object, animal and scene images. We found reduced neural decoding of face identity around the N170 window in PS, and conjointly revealed normal non-face identification in this patient. We used Representational Similarity Analysis (RSA) to correlate human EEG representations with those of deep neural network (DNN) models of vision and caption-level semantics, offering a window into the neural computations at play in patient PS's deficits. Brain representational dissimilarity matrices (RDMs) were computed for each participant at 4 ms steps using cross-validated classifiers. PS's brain RDMs showed significant reliability across sessions, indicating meaningful measurements of brain representations with RSA even in the presence of significant lesions. Crucially, computational analyses were able to reveal PS's representational deficits in high-level visual and semantic brain computations. Such multi-modal data-driven characterisations of prosopagnosia highlight the complex nature of processes contributing to face recognition in the human brain.

show abstract

Degrees of algorithmic equivalence between the brain and its DNN models

Cited by 22 publications

References 84 publications

Unraveling the complexity of rat object vision requires a full convolutional network - and beyond

Unraveling the complexity of rat object vision requires a full convolutional network - and beyond

The Brain Computes Dynamic Facial Movements for Emotion Categorization Using a Third Pathway

Neural computations in prosopagnosia

Contact Info

Product

Resources

About