Partitioning natural face image variability emphasises within-identity over between-identity representation for understanding accurate recognition

Abstract

Accurately recognising faces enables social interactions. In recent years it has become clear that people's accuracy differs markedly depending on viewer's familiarity with a face and their individual skill, but the cognitive and neural bases of these accuracy differences are not understood. We examined cognitive representations underlying these accuracy differences by measuring similarity ratings to natural facial image variation. Natural variation was sampled from uncontrolled images on the internet to reflect the appearance of faces as they are encountered in daily life. Using image averaging, and inspired by the computation of Analysis of Variance, we partitioned this variation into differences between faces (betweenidentity variation) and differences between photos of the same face (within-identity variation). This allowed us to compare modulation of these two sources of variation attributable to: (i) a person's familiarity with a face and, (ii) their face recognition ability. Contrary to prevailing accounts of human face recognition and perceptual learning, we found that modulation of within-identity variation -rather than between-identity variation -was associated with high accuracy. First, familiarity modulated similarity ratings to withinidentity variation more than to between-face variation. Second, viewers that are extremely accurate in face recognition -'super-recognisers' -differed from typical perceivers mostly in their ratings of within-identity variation, compared to between-identity variation. In a final computational analysis, we found evidence that transformations of between-and withinidentity variation make separable contributions to perceptual expertise in face recognition. We conclude that inter-and intra-individual accuracy differences primarily arise from differences in the representation of within-identity image variation.

Unfamiliar face matching ability predicts the slope of face learning

“…On the surface, our data contrast with a recent examination of the effect of familiarity on perceivers’ similarity judgements of within- vs. between-identity face pairs 70 —at least when tested with highly familiar faces. Familiarity increased similarity ratings of within-identity pairs more than it decreased similarity ratings of between-identity pairs—an effect that was most pronounced for super recognizers.…”

A signal detection–based confidence–similarity model of face matching.

“…The two sources of variability in face images are crucial for understanding how we learn new faces (Kramer et al, 2017) and, particularly, what are the transformations that novel faces undergo in our brain on their way to becoming familiar (Kramer et al, 2018). To examine this question, White et al (2022) employed an analysis of variance (ANOVA)-like approach as a means of delineating the This document is copyrighted by the American Psychological Association or one of its allied publishers.…”

“…The UVSD model is based on the assumption that face perception is affected by both within-identities and between-identities sources of variability (White et al, 2022). Another assumption is that with unfamiliar faces, the standard deviation of the match distribution is expected to be larger than the standard deviation of the mismatch distribution.…”

“…Another assumption is that with unfamiliar faces, the standard deviation of the match distribution is expected to be larger than the standard deviation of the mismatch distribution. This assumption is based on behavioral (White et al, 2022) and computer vision (Moses et al, 1994) studies. It should be noted though that in the context of an experiment, the relations between the two sources of variability can be deliberately or unknowingly altered by the experimenter.…”

Twenty years of investigation with the case of prosopagnosia PS to understand human face identity recognition. Part I: Function

“…Second, the same face identity can vary substantially under different viewing conditions and over time, to the point where two views of the same face identity often differ physically to a larger extent than two different facial identities (i.e., 'within-person variability' could be greater than 'between-person variability'; e.g., Burton et al, 2016;White et al, 2022). This is why recognition of several instances of a facial identity as belonging to the same person does not obey classical rules of categorization, i.e., it cannot be based on merely counting ("computing") sufficient and/or necessary features (Smith & Medin, 1981;Murphy, 2002), and FIR requires a high-level of generalization of a specific response across different facial views.…”