Mixed emotions: Sensitivity to facial variance in a crowd of faces

Abstract

The visual system automatically represents summary information from crowds of faces, such as the average expression. This is a useful heuristic insofar as it provides critical information about the state of the world, not simply information about the state of one individual. However, the average alone is not sufficient for making decisions about how to respond to a crowd. The variance or heterogeneity of the crowd--the mixture of emotions--conveys information about the reliability of the average, essential for determining whether the average can be trusted. Despite its importance, the representation of variance within a crowd of faces has yet to be examined. This is addressed here in three experiments. In the first experiment, observers viewed a sample set of faces that varied in emotion, and then adjusted a subsequent set to match the variance of the sample set. To isolate variance as the summary statistic of interest, the average emotion of both sets was random. Results suggested that observers had information regarding crowd variance. The second experiment verified that this was indeed a uniquely high-level phenomenon, as observers were unable to derive the variance of an inverted set of faces as precisely as an upright set of faces. The third experiment replicated and extended the first two experiments using method-of-constant-stimuli. Together, these results show that the visual system is sensitive to emergent information about the emotional heterogeneity, or ambivalence, in crowds of faces.

Comparing set summary statistics and outlier pop out in vision

“…The results of all four groups are similar, with only minor differences in speed or accuracy, but all showing the same essential effects described in the preceding paragraph. Our results should be considered in light of contrasting findings suggesting that set mean perception and outlier ''pop-out'' accuracy decrease with set variance (e.g., see Dakin 2001;Rosenholtz, 2001;Haberman et al, 2015). For set mean, the type of elements being averaged may be important; line, circle, or face elements may be different than Gabor patterns, which lack clear borders and may cluster when crowded so that stimulus noise becomes critical.…”

“…Observer reports of set statistics include set feature variance or range, as shown directly (Morgan, Chubb, & Solomon, 2008;Solomon, 2010;Haberman, Lee & Whitney, 2015) and as evidenced by their reporting presence or absence of items depending on their being within or outside the range, respectively (Pollard, 1984;Dakin & Watt 1997). Recently, Khayat and Hochstein (2018) reported that observers perceive not only set mean but also set range, implicitly, automatically, and on-the-fly, on a trial-by-trial basis.…”

Ensemble perception of facial attractiveness

“…For facial features, Haberman, Lee, and Whitney (2015) found observers able to accurately estimate the variance in facial expressions of a group, establishing the visual system's sensitivity to variance in high-level domains. We thus asked whether computation of mean facial attractiveness would be affected by set variance.…”

Roles of saliency and set size in ensemble averaging

“…It has been previously demonstrated that people can efficiently extract various statistical information form briefly presented multiple items. Summary statistics such as mean and variance can be extracted about various features of individual items from basic sensory dimensions, like orientation (Alvarez & Oliva, 2009;Dakin & Watt, 1997;Morgan, Chubb, & Solomon, 2008;Parkes, Lund, Angelucci, Solomon, & Morgan, 2001;Suárez-Pinilla, Seth, & Roseboom, 2018), size (Ariely, 2001;Chong & Treisman, 2003;Khvostov & Utochkin, 2019;Tokita, Ueda, & Ishiguchi, 2016), color (Bronfman, Brezis, Jacobson, & Usher, 2014;Gardelle & Summerfield, 2011;Maule & Franklin, 2015), to quite complex and high-level dimensions, like facial expression (Haberman, Lee, & Whitney, 2015;Haberman & Whitney, 2007) or animacy (Leib, Kosovicheva, & Whitney, 2016). Interestingly, the efficiency and accuracy of such ensemble representation does not suffer (Ariely, 2001;Chong & Treisman, 2005;Fouriezos, Rubenfeld, & Capstick, 2008;Haberman, Harp, & Whitney, 2009; or even benefits (Chong, Joo, Emmmanouil, & Treisman, 2008;Robitaille & Harris, 2011) from increasing set size, whereas our ability to report individual items quickly degrades with set size (Ariely, 2001;Haberman & Whitney, 2007).…”