Feature-based encoding of face identity by single neurons in the human amygdala and hippocampus

Lin

et al. 2021

Preprint

Self Cite

People instantaneously evaluate faces with significant agreement on evaluations of social traits. However, the neural basis for such rapid spontaneous face evaluation remains largely unknown. Here, we recorded from 490 neurons in the amygdala and hippocampus in 5 neurosurgical patients and show that amygdala and hippocampal neurons encode a social trait space. We further investigated the temporal evolution and modulation on the social trait representation, and we employed encoding and decoding models to reveal the critical social traits for the trait space. We also recorded from another 259 neurons and replicated our findings using different social traits. Lastly, the neuronal social trait space may have a behavioral consequence likely involved in the abnormal processing of social information in autism. Together, our results suggest that there exists a neuronal population code for a comprehensive social trait space in the human amygdala and hippocampus that underlie spontaneous first impressions.

Section: Resultssupporting

confidence: 53%

A neuronal social trait space for first impressions in the human amygdala and hippocampus

Lin

et al. 2021

Preprint

Self Cite

“…Such difference in function between the amygdala and hippocampus is consistent with our previous finding showing that only the amygdala but not the hippocampus encode perceived facial emotions (Wang et al 2014). However, the amygdala and hippocampus play a similar role in encoding face identities (Cao et al 2020), visual selectivity (Kreiman et al 2000, Wang et al 2018), memory (Rutishauser et al 2010), and visual attention (Wang et al 2018). Furthermore, although some faces in the CelebA stimuli have facial expressions (because the celebrities posed when taking the photos) and eye movement can be biased by facial expressions (Scheller et al 2012), we observed similar results in an unpublished study using emotionally neutral FaceGen model faces.…”

Section: Discussionmentioning

confidence: 99%

“…Only units with an average firing rate of at least 0.15 Hz (entire task) were considered (Cao et al 2020). Fixations were aligned to fixation onset and saccades were aligned to saccade onset.…”

Section: Methodsmentioning

confidence: 99%

Encoding of facial features by single neurons in the human amygdala and hippocampus

Brandmeir

et al. 2020

Preprint

Self Cite

The human amygdala and hippocampus play a key role in face processing. However, it has been unknown how the neurons in the human amygdala and hippocampus encode facial feature information and directs eye movements to salient facial features such as the eyes and mouth. In this study, we identified a population of neurons that differentiated fixations on the eyes vs. mouth. The response of these feature-selective neurons was not dependent on fixation order, and eye-preferring and mouth-preferring neurons were not of different neuronal types. We found another population of neurons that differentiated saccades to the eyes vs. mouth. Population decoding confirmed our results and further revealed the temporal dynamics of face feature coding. Interestingly, we found that the amygdala and hippocampus played a different role in encoding face features. Lastly, we revealed two functional roles of feature-selective neurons that they encoded the salient region for face recognition and they encoded perceived social trait judgment. Together, we revealed and characterized a new class of neurons that encoded facial features. These neurons may play an important role in social perception and recognition of faces.

“…The present study continues from our recent work showing feature-based encoding of face identities in the human MTL using DNN-extracted visual features [24] (an encoding-related study). The motivation for the present study is largely two-fold.…”

Section: Introductionmentioning

confidence: 62%

“…Patients performed a one-back task (Fig. 5A; accuracy = 75.7±5.28% [mean±SD across sessions]) and they could well recognize the faces [24]. The responses of 46/490 neurons (9.39%) differed between different face identities in a window 250-1000 ms following stimulus onset and these neurons were the real human identityselective neurons.…”

Section: Establishing the Relationship Between Artificial Dnn Neurons And Real Human Neuronsmentioning

confidence: 99%

Face identity selectivity in the deep neural network and human brain

Wang

et al. 2021

Preprint

Self Cite

A central challenge in face perception research is to understand how neurons encode various face identities. However, this challenge has not been met largely due to the lack of simultaneous access to the activity of the entire face processing neural network as well as the lack of a comprehensive multifaceted model that is able to characterize a large number of facial features. In this study, we address this challenge by conducting in silico experiments using a deep neural network (DNN) capable of face recognition with a diverse array of stimuli. We identified a subset of DNN neurons selective to face identities, and these identity-selective neurons demonstrated generalized discriminability to novel faces not involved in the training and in many different styles. Visualization of the network explained the response of the DNN neurons and manipulation of the network confirmed the importance of identity-selective neurons in face recognition. Importantly, using our human single-neuron recordings, we directly compared the response of artificial neurons with 490 real human neurons to the same stimuli and found that artificial neurons did share a similar representation of facial features as human neurons. We also observed a novel region-based feature coding mechanism in DNN neurons as in human neurons, which may explain how the DNN performs face recognition. Together, by directly linking between artificial and human neurons, our results shed light on how human neurons encode face identities.