Multimodal evidence on shape and surface information in individual face processing

Abstract

The significance of shape and surface information for face perception is well established, yet their relative contribution to recognition and their neural underpinnings await clarification. Here, we employ image reconstruction to retrieve, assess and visualize such information using behavioral, electroencephalography and functional magnetic resonance imaging data.Our results indicate that both shape and surface information can be successfully recovered from each modality but that the latter is better recovered than the former, consistent with its key role for face representations. Further, shape and surface information exhibit similar spatiotemporal profiles, rely on the extraction of specific visual features, such as eye shape or skin tone, and reveal a systematic representational structure, albeit with more cross-modal consistency for shape than surface.Thus, the present results help elucidate the representational basis of individual face recognition while, methodologically, they showcase the utility of image reconstruction and clarify its reliance on diagnostic visual information..

Image reconstruction reveals the impact of aging on face perception.

“…Regarding the nature of the information retrieved above, we note that surface information was systematically recovered more accurately than shape information in all participants, irrespective of age. This finding is consistent with previous results demonstrating better sensitivity to surface than shape in human participants but not in a theoretical observer, which was capable of retrieving both types of information with equivalent levels of success (Nemrodov et al, 2019). More generally, these results are consistent with the dominant role of surface properties in face recognition (Andrews et al, 2016; Michel et al, 2013).…”

“…We adopted a recent approach to facial image reconstruction (Nemrodov et al, 2019; Nestor et al 2020), which relies on face de/recomposition into shape and surface information. This procedure separately recovers shape, defined as the position of a set of fiducial points (i.e., 82 points × 2 in-plane coordinates positioned at the key feature locations, such as the corners of eyes and mouth), and surface, as given by pixel intensity values in CIEL*a*b* after warping face images to a standard shape template.…”

“…Reconstruction accuracy was measured objectively based on Euclidean pointwise similarity for shape, or pixelwise intensity similarity for surface, between reconstructions and target faces (Nemrodov et al, 2019). Specifically, accuracy was calculated as the proportion of instances for which the reconstruction was more similar to its target than any other face.…”

“…In the present study, face space constructs were estimated separately for each participant from behavioral estimates of pairwise face similarity. Then, facial shape and surface features were derived from the structure of face space (Nemrodov et al, 2019) and combined to recreate the overall appearance of individual faces as perceived by young and older adults. The reconstruction was evaluated objectively with regard to its shape and surface properties (Experiment 1), as well as experimentally by an independent group of participants (Experiment 2).…”