Efficient inverse graphics in biological face processing

Yildirim, Ilker; Belledonne, Mario; Freiwald, Winrich A.; Tenenbaum, Josh

doi:10.1126/sciadv.aax5979

Cited by 75 publications

(78 citation statements)

References 58 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Here, we sought to overcome these limitations by adopting the underlying theoretical idea (wiring cost minimization), but building upon recent advances in ANN models (1,12,36). Notably, that prior deep ANN modeling work has already qualitatively demonstrated the presence of at least some "face neurons" within model IT (36) and more recent studies have demonstrated the existence of face-selective units in deep ANNs (37)(38)(39). However, the correspondence of face processing in ANNs and the primate ventral stream has not been tested systematically.…”

Section: Category Choicementioning

confidence: 99%

Topographic deep artificial neural networks reproduce the hallmarks of the primate inferior temporal cortex face processing network

Lee

Margalit

Jozwik

et al. 2020

Preprint

View full text Add to dashboard Cite

A salient characteristic of monkey inferior temporal (IT) cortex is the IT face processing network. Its hallmarks include: “face neurons” that respond more to faces than non-face objects, strong spatial clustering of those neurons in foci at each IT anatomical level (“face patches”), and the preferential interconnection of those foci. While some deep artificial neural networks (ANNs) are good predictors of IT neuronal responses, including face neurons, they do not explain those face network hallmarks. Here we ask if they might be explained with a simple, metabolically motivated addition to current ANN ventral stream models. Specifically, we designed and successfully trained topographic deep ANNs (TDANNs) to solve real-world visual recognition tasks (as in prior work), but, in addition, we also optimized each network to minimize a proxy for neuronal wiring length within its IT layers. We report that after this dual optimization, the model IT layers of TDANNs reproduce the hallmarks of the IT face network: the presence of face neurons, clusters of face neurons that quantitatively match those found in IT face patches, connectivity between those patches, and the emergence of face viewpoint invariance along the network hierarchy. We find that these phenomena emerge for a range of naturalistic experience, but not for highly unnatural training. Taken together, these results show that the IT face processing network could be a consequence of a basic hierarchical anatomy along the ventral stream, selection pressure on the visual system to accomplish general object categorization, and selection pressure to minimize axonal wiring length.

show abstract

Section: Category Choicementioning

confidence: 99%

Topographic deep artificial neural networks reproduce the hallmarks of the primate inferior temporal cortex face processing network

Lee

Margalit

Jozwik

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Explicit experimental hypotheses are directly tied to the parameterisation of stimuli by generative models and vice versa. For example, we explicitly tested that a parameterisation of faces in terms of their 3D shape and RGB texture could mediate human and DNN behaviour in the task (Zhan et al, 2019a;Yildirim et al, 2020). Our study thereby contributes to the debate of the degree to which convolutional DNNs can make use of shape information in images (Kubilius et al, 2016;Xu et al, 2018;Baker et al, 2018;Geirhos et al, 2019;Hermann & Kornblith, 2019;Doerig et al, 2020).…”

Section: Hypothesis-driven Research Using Generative Modelsmentioning

confidence: 91%

Grounding deep neural network predictions of human categorization behavior in understandable functional features: The case of face identity

Daube¹,

Xu²,

Zhan³

et al. 2020

Preprint

View full text Add to dashboard Cite

Deep neural networks (DNNs) can resolve real-world tasks with apparent human-like performance and could therefore offer accessible information-processing models of the brain. However, interpreting these models remains a challenge. Here, we address this challenge by leveraging the experimental control of a generative model of the stimuli, in the form of faces that vary in 3D shape and texture. We asked human participants to rate how similar randomly generated faces were to four familiar colleagues. We then predicted these human responses from the shape and texture parameters of the generative model and from the activations of four DNNs trained on different optimisation objectives. With information- theoretic redundancy and reverse correlation, we then grounded behavioural prediction performance into interpretable functional shape and texture features. Our study addresses a critical prerequisite to evaluating the similarity between the brain and its DNN models: both must process the same functional features in the task.

show abstract

“…This meant simultaneously satisfying many functional requirements (Table 1), and multiple tasks ( Figure 5) instead of optimizing for a single query-dependent objective. It is an encouraging sign that many more recent models (Linsley et al, 2018;Kietzmann et al, 2019;Yildirim et al, 2020) have started incorporating insights from neuroscience toward building a unified general model for vision. We now consider the different properties of RCN from the viewpoint of triangulation strategy where we describe their neuroscience origins, their correspondence with the world, and their computational underpinnings.…”

Section: What Kind Of Visual Generative Model Is Suitable For Common mentioning

confidence: 98%

From CAPTCHA to Commonsense: How Brain Can Teach Us About Artificial Intelligence

George¹,

Lázaro-Gredilla²,

Guntupalli³

2020

Front. Comput. Neurosci.

View full text Add to dashboard Cite

Despite the recent progress in AI powered by deep learning in solving narrow tasks, we are not close to human intelligence in its flexibility, versatility, and efficiency. Efficient learning and effective generalization come from inductive biases, and building Artificial General Intelligence (AGI) is an exercise in finding the right set of inductive biases that make fast learning possible while being general enough to be widely applicable in tasks that humans excel at. To make progress in AGI, we argue that we can look at the human brain for such inductive biases and principles of generalization. To that effect, we propose a strategy to gain insights from the brain by simultaneously looking at the world it acts upon and the computational framework to support efficient learning and generalization. We present a neuroscience-inspired generative model of vision as a case study for such approach and discuss some open problems about the path to AGI.

show abstract

Efficient inverse graphics in biological face processing

Cited by 75 publications

References 58 publications

Topographic deep artificial neural networks reproduce the hallmarks of the primate inferior temporal cortex face processing network

Topographic deep artificial neural networks reproduce the hallmarks of the primate inferior temporal cortex face processing network

Grounding deep neural network predictions of human categorization behavior in understandable functional features: The case of face identity

From CAPTCHA to Commonsense: How Brain Can Teach Us About Artificial Intelligence

Contact Info

Product

Resources

About