Unraveling the distributed neural code of facial identity through spatiotemporal pattern analysis

Abstract

Face individuation is one of the most impressive achievements of our visual system, and yet uncovering the neural mechanisms subserving this feat appears to elude traditional approaches to functional brain data analysis. The present study investigates the neural code of facial identity perception with the aim of ascertaining its distributed nature and informational basis. To this end, we use a sequence of multivariate pattern analyses applied to functional magnetic resonance imaging (fMRI) data. First, we combine information-based brain mapping and dynamic discrimination analysis to locate spatiotemporal patterns that support face classification at the individual level. This analysis reveals a network of fusiform and anterior temporal areas that carry information about facial identity and provides evidence that the fusiform face area responds with distinct patterns of activation to different face identities. Second, we assess the information structure of the network using recursive feature elimination. We find that diagnostic information is distributed evenly among anterior regions of the mapped network and that a right anterior region of the fusiform gyrus plays a central role within the information network mediating face individuation. These findings serve to map out and characterize a cortical system responsible for individuation. More generally, in the context of functionally defined networks, they provide an account of distributed processing grounded in information-based architectures.T he neural basis of face perception is the focus of extensive research as it provides key insights both into the computational architecture of visual recognition (1, 2) and into the functional organization of the brain (3). A central theme of this research emphasizes the distribution of face processing across a network of spatially segregated areas (4-10). However, there remains considerable disagreement about how information is represented and processed within this network to support tasks such as individuation, expression analysis, or high-level semantic processing.One influential view proposes an architecture that maps different tasks to distinct, unique cortical regions (6) and, as such, draws attention to the specificity of this mapping (11)(12)(13)(14)(15)(16)(17)(18)(19)(20). As a case in point, face individuation (e.g., differentiating Steve Jobs from Bill Gates across changes in expression) is commonly mapped onto the fusiform face area (FFA) (6, 21). Although recent studies have questioned this role of the FFA (14, 15), overall they agree with this task-based architecture as they single out other areas supporting individuation.However, various distributed accounts have also been considered. One such account ascribes facial identity processing to multiple, independent regions. Along these lines, the FFA's sensitivity to individuation has been variedly extended to areas of the inferior occipital gyrus (5), the superior temporal sulcus (12), and the temporal pole (22). An alternative scenario is that identity is encode...

“Hearing faces and seeing voices”: Amodal coding of person identity in the human brain

“…This result is consistent with previous reports of face identity classification based on local fMRI clusters in fusiform areas [19][20][21][22] suggesting that activity in these areas of the core face processing network already contain sufficient identity information to discriminate between facial identities. We did not observe face classification in more anterior areas of the anterior temporal lobe reported by some studies 19,20,23 . Voice classification based on local fMRI clusters is, to our knowledge, a novel result.…”

“…Other studies suggest that amodal representations could rather emerge from cross-talk interactions between modality-specific areas 1 : voice and face-sensitive areas are not only connected via direct anatomical projections 15 but also functionally connected during familiar voice recognition 16 . Multi-voxel pattern analyses (MVPA) offer a powerful means of extracting information contained in distributed fMRI activity 17,18 : their enhanced sensitivity compared to classical univariate fMRI analyses has contributed to clarifying the neural correlates of unimodal face [19][20][21][22][23] or voice 24,25 identity processing. To our knowledge MVPA have not been used yet to examine the cerebral bases of multimodal person identification although they have provided exciting insights in the integration of emotional information across senses.…”

Quantifying interindividual variability and asymmetry of face-selective regions: A probabilistic functional atlas

“…Among them, the most reliable regions are located in the occipitotemporal cortex. Specifically, one FSR is in the inferior occipital gyrus (IOG, or occipital face area, OFA) (Gauthier et al, 2000;Liu et al, 2010); three FSRs are in the fusiform gyrus (FG, or fusiform face area, FFA), located in the posterior, middle and anterior parts of the FG (i.e., pFFA, mFFA, and aFFA) (Engell and McCarthy, 2013;Gauthier et al, 1999;Kanwisher et al, 1997;Nestor et al, 2011;Weiner and Grill-Spector, 2010), and three FSRs are in the superior temporal sulcus (STS), located at the posterior continuation of the STS, the posterior STS, and the anterior STS (i.e., pcSTS, pSTS, and aSTS) (Pinsk et al, 2009;Pitcher et al, 2011a;Puce et al, 1998). These regions are thought to process different aspects of faces : the region located in the IOG is involved in early perception of facial features; those regions located in the FG analyze the invariant aspects of faces that underlie recognition of individuals (Kanwisher et al, 1997;McCarthy et al, 1997;Pitcher et al, 2011b) and those regions located in the STS process the changeable aspects of faces such as expressions (Phillips et al, 1997), direction of eye gaze, and lip movements, for facilitating social communication (Allison et al, 2000;Hoffman and Haxby, 2000;Lahnakoski et al, 2012;Puce et al, 1998).…”

Spatiotemporal dynamics of similarity-based neural representations of facial identity

“…Progress has been made in elucidating the neural mechanisms underlying the discrimination of individual face identities in humans. Using fMRI, these studies demonstrate that individual face identities are represented by spatially distributed patterns of neural activity within occipitotemporal cortex (3)(4)(5)(6)(7)(8)(9)(10)(11)(12)(13). Because of the poor temporal resolution in fMRI studies (typically around 2 s), however, our understanding of the neural basis of discrimination among complex visual patterns in humans remains limited.…”