The Face Inversion Effect in Deep Convolutional Neural Networks

Fang, Tong; Xie, Hailun; Song, Yiying; Hu, Siyuan; Liu, Jia

doi:10.3389/fncom.2022.854218

Cited by 31 publications

(14 citation statements)

References 38 publications

(56 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similarly, the distance between the embeddings of these face stimuli is larger when presented upright than inverted in the higher layers of a face-trained but not object-trained DCNN. A face inversion effect in a DCNN was also recently reported by Tian and colleagues 30 . Another evidence for the specificity of the face processing system of humans and DCNNs, to the stimuli it was trained with, is the other race effect.…”

Section: Discussionsupporting

confidence: 70%

Deep learning models of perceptual expertise support a domain-specific account

Yovel

Grosbard

Abudarham

2022

Preprint

View full text Add to dashboard Cite

Perceptual expertise is an acquired skill that enables fine discrimination of members of a homogenous category. The question of whether perceptual expertise is mediated by general-expert or domain-specific processing mechanisms has been hotly debated for decades in human behavioral and neuroimaging studies. To decide between these two hypotheses, most studies examined whether expertise for different domains is mediated by the same mechanisms used for faces, for which most humans are expert. Here we used deep convolutional neural networks (DCNNs) to test whether perceptual expertise is best achieved by computations that are optimized for face or object classification. We re-trained a face-trained and an object-trained DCNNs to classify birds at the sub-ordinate or individual-level of categorization. The face-trained DCNN required deeper retraining to achieve the same level of performance for bird classification as an object-trained DCNN. These findings indicate that classification at the subordinate- or individual-level of categorization does not transfer well between domains. Thus, fine-grained classification is best achieved by using domain-specific rather than domain-general computations.

show abstract

Section: Discussionsupporting

confidence: 70%

Deep learning models of perceptual expertise support a domain-specific account

Yovel

Grosbard

Abudarham

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Abudarham and colleagues (2021) showed that face-trained but not object-trained DCNNs are sensitive to the same view-invariant, critical facial features used by humans for face recognition. Recent studies have also shown that face-trained DCNNs generate a face inversion effect, similar to the well-established effect shown in humans 8–10 . Finally, face recognition DCNNs show better classification for the race of faces they are trained with 11–13 , similar to the other-race effect, which has been extensively investigated in humans 14,15 .…”

Section: Introductionsupporting

confidence: 71%

“…A pure semantic DNN increases the total explained variance in human memory to 67%. Overall, these findings show that visualsemantic and semantic DNNs significantly improve the prediction of human representations of familiar faces, beyond the pure visual algorithm that has been so far used to model human face representations 3,4,[6][7][8]10,11,[28][29][30] .…”

Section: As Shown Inmentioning

confidence: 85%

See 1 more Smart Citation

Deep learning algorithms reveal a new visual-semantic representation of familiar faces in human perception and memory

Shoham

Grosbard

Patashnik

et al. 2022

Preprint

View full text Add to dashboard Cite

Recent studies show significant similarities between the representations humans and deep neural networks (DNNs) generate for faces. However, two critical aspects of human face recognition are overlooked by these networks. First, human face recognition is mostly concerned with familiar faces, which are encoded by visual and semantic information, while current DNNs solely rely on visual information. Second, humans represent familiar faces in memory, but representational similarities with DNNs were only investigated for human perception. To address this gap, we combined visual (VGG-16), visual-semantic (CLIP), and natural language processing (NLP) DNNs to predict human representations of familiar faces in perception and memory. The visual-semantic network substantially improved predictions beyond the visual network, revealing a new visual-semantic representation in human perception and memory. The NLP network further improved predictions of human representations in memory. Thus, a complete account of human face recognition should go beyond vision and incorporate visual-semantic, and semantic representations.

show abstract

“…To address these limitations in human studies, in the current study, we used deep convolutional neural networks (DCNNs) as computational models of perceptual expertise. DCNNs are brain-inspired algorithms that reach humanlevel performance and generate human-like representations for objects and faces [31][32][33][34][35][36]. These models can be trained to classify images from different domains at different levels of categorization [23].…”

Section: Introductionmentioning

confidence: 99%

Deep learning models challenge the prevailing assumption that face-like effects for objects of expertise support domain-general mechanisms

2023

View full text Add to dashboard Cite

The question of whether task performance is best achieved by domain-specific, or domain-general processing mechanisms is fundemental for both artificial and biological systems. This question has generated a fierce debate in the study of expert object recognition. Because humans are experts in face recognition, face-like neural and cognitive effects for objects of expertise were considered support for domain-general mechanisms. However, effects of domain, experience and level of categorization, are confounded in human studies, which may lead to erroneous inferences. To overcome these limitations, we trained deep learning algorithms on different domains (objects, faces, birds) and levels of categorization (basic, sub-ordinate, individual), matched for amount of experience. Like humans, the models generated a larger inversion effect for faces than for objects. Importantly, a face-like inversion effect was found for individual-based categorization of non-faces (birds) but only in a network specialized for that domain. Thus, contrary to prevalent assumptions, face-like effects for objects of expertise do not support domain-general mechanisms but may originate from domain-specific mechanisms. More generally, we show how deep learning algorithms can be used to dissociate factors that are inherently confounded in the natural environment of biological organisms to test hypotheses about their isolated contributions to cognition and behaviour.

show abstract

The Face Inversion Effect in Deep Convolutional Neural Networks

Cited by 31 publications

References 38 publications

Deep learning models of perceptual expertise support a domain-specific account

Deep learning models of perceptual expertise support a domain-specific account

Deep learning algorithms reveal a new visual-semantic representation of familiar faces in human perception and memory

Deep learning models challenge the prevailing assumption that face-like effects for objects of expertise support domain-general mechanisms

Contact Info

Product

Resources

About