Adjudicating between face-coding models with individual-face fMRI responses

Abstract

The perceptual representation of individual faces is often explained with reference to a norm-based face space. In such spaces, individuals are encoded as vectors where identity is primarily conveyed by direction and distinctiveness by eccentricity. Here we measured human fMRI responses and psychophysical similarity judgments of individual face exemplars, which were generated as realistic 3D animations using a computer-graphics model. We developed and evaluated multiple neurobiologically plausible computational models, each of which predicts a representational distance matrix and a regional-mean activation profile for 24 face stimuli. In the fusiform face area, a face-space coding model with sigmoidal ramp tuning provided a better account of the data than one based on exemplar tuning. However, an image-processing model with weighted banks of Gabor filters performed similarly. Accounting for the data required the inclusion of a measurement-level population averaging mechanism that approximates how fMRI voxels locally average distinct neuronal tunings. Our study demonstrates the importance of comparing multiple models and of modeling the measurement process in computational neuroimaging.

The neural representation of facial-emotion categories reflects conceptual structure

“…50 and 117). Because multiple-regression RSA assumes a linear combination of multiple predictor DMs, these analyses require a dissimilarity measure that sums linearly; thus, squared Euclidean distance is an appropriate measure (118). However, because squared Euclidean distances of normalized pattern vectors are equivalent (i.e., linearly proportional) to Pearson correlation distances (119), we report results in Pearson correlation distance for ease of understanding and for greater intuitiveness.…”

Spatio-temporal dynamics of face perception

“…Changeable aspects, (or motion: Bernstein & Yovel, 2015) such as expressions, are mainly processed in the dorsal stream, especially in the superior temporal sulcus (STS; Greening, Mitchell, & Smith, 2018;Said, Haxby, & Todorov, 2011;Zhang et al, 2016). In contrast, invariant aspects, such as identity, are processed in the ventral stream from part-based processing in the occipital face area (OFA; Atkinson & Adolphs, 2011;Henriksson, Mur, & Kriegeskorte, 2015;Pitcher, Walsh, & Duchaine, 2011), to the fusiform face area (FFA; Anzellotti, Fairhall, & Caramazza, 2014;Carlin & Kriegeskorte, 2017;Dobs, Schultz, Bülthoff, & Gardner, 2018;Kanwisher & Yovel, 2006), and finally to highestlevel, viewpoint-invariant processing in the ventral anterior temporal lobe (vATL; Anzellotti & Caramazza, 2016;Anzellotti et al, 2014;Collins & Olson, 2015;Kriegeskorte, Formisano, Sorger, & Goebel, 2007). Hierarchical processing is supported by single-cell recordings from macaques that have found viewpoint-specific coding of face identities in the middle lateral and middle fungus, mirror-symmetrical processing in the anterior lateral patch, and finally almost viewpoint-invariant identity representations in the anterior medial patch (Chang & Tsao, 2017;Freiwald & Tsao, 2010).…”

Adaptation aftereffects reveal how categorization training changes the encoding of face identity

“…The second channel, represented by the solid yellow curves, responds more strongly to faces that are more similar to the anti-parent identity. A wealth of psychophysical (e.g., Leopold et al, 2001;McKone et al, 2014;Rhodes & Jeffery, 2006), neurophysiological (e.g., Freiwald et al, 2009;Giese & Leopold, 2005;Leopold et al, 2006), and neuroimaging (e.g., Carlin & Kriegeskorte, 2017;Loffler et al, 2005) data support such coding for face identity (but see Ross et al, 2013). However, we assume that a more complete model would involve explicit modeling of the full face space and the multidimensional tuning of many channels distributed in such space (e.g., Giese & Leopold, 2005;Ross et al, 2013).…”

“…In addition, adaptation to a face produces a stronger aftereffect along the morphing dimension going to its corresponding anti-face (through the average) than along a dimension going to a second face, even when dissimilarity is matched (Rhodes & Jeffery, 2006). Finally, evidence for such a code has been found in the human fusiform face area (Carlin & Kriegeskorte, 2017;Loffler et al, 2005) as well as in monkey inferior temporal cortex (Freiwald et al, 2009;Leopold et al, 2006). In line with such findings, it is commonly assumed that studies of face encoding target representations stored in face-selective areas within inferior temporal cortex, at the latest stages of processing in the visual ventral stream (Giese & Leopold, 2005;Jiang et al, 2006).…”