Elucidating the Neural Representation and the Processing Dynamics of Face Ensembles

Abstract

Extensive behavioral work has documented the ability of the human visual system to extract summary representations from face ensembles (e.g., the average identity of a crowd of faces). Yet, the nature of such representations, their underlying neural mechanisms, and their temporal dynamics await elucidation. Here, we examine summary representations of facial identity in human adults (of both sexes) with the aid of pattern analyses, as applied to EEG data, along with behavioral testing. Our findings confirm the ability of the visual system to form such representations both explicitly and implicitly (i.e., with or without the use of specific instructions). We show that summary representations, rather than individual ensemble constituents, can be decoded from neural signals elicited by ensemble perception, we describe the properties of such representations by appeal to multidimensional face space constructs, and we visualize their content through neural-based image reconstruction. Further, we show that the temporal profile of ensemble processing diverges systematically from that of single faces consistent with a slower, more gradual accumulation of perceptual information. Thus, our findings reveal the representational basis of ensemble processing, its fine-grained visual content, and its neural dynamics.

A multivariate investigation of visual word, face, and ensemble processing: Perspectives from EEG‐based decoding and feature selection

“…Further, diagnosticity does not ramp up around 170 ms, as for words and single faces, but instead peaks later within a 350–450 ms interval. This temporal profile presumably reflects a slower process of information accumulation underlying the derivation of a summary representation (Haberman, Harp, & Whitney, ; Roberts et al, ). Arguably, this demonstrates how face ensemble and single‐face processing diverge systematically in their spatiotemporal dynamics in response to distinct perceptual demands.…”

“…Of particular relevance here, recent work has found evidence for the ability of EEG pattern analyses to support the decoding of facial identity (Ambrus et al, ; Nemrodov, Niemeier, Patel, & Nestor, ), visual words (Chan, Halgren, Marinkovic, & Cash, ; Ling, Lee, Armstrong, & Nestor, ), and face ensembles (Roberts, Cant, & Nestor, ) suggesting a promising strategy to address cross‐category commonalities in neural dynamics through multivariate analyses.…”

“…Face ensemble processing was assessed based on the data of 14 participants (10 females, age: 19–25) who performed a go/no‐go gender categorization task (Roberts et al, ). Each of four base ensembles, presented on no‐go trials and used in the current analysis, consisted of six different male faces displayed in a circular arrangement.…”

“…Briefly, our investigation applied the strategy above to three EEG data sets dedicated to the study of facial identity processing (Nemrodov et al, ), word recognition (Ling et al, ), and ensemble perception (Roberts et al, ). Our examination revealed robust patterns of feature diagnosticity within and across participants, similar profiles of spatiotemporal information guiding the decoding of single faces and words, but not of face ensembles, as well as systematic boosts in decoding accuracy accompanied by significant reductions of feature sets for each category.…”

EEG Signal Classification: An Application to the Emotion Related Brain Anticipatory Activity

“…temporal features), corresponding to adjacent temporal windows, is extracted. The temporal features are fed into either a "dynamic" classifiers, such as Hidden Markow Model (HMM) [21], or an ordered sequence of "static" classifiers { } =1 [36][37][38][39]. The former fully takes in account the signal's temporal variability, since it uses the entire sequence during the training phase.…”