Facial identity and facial expression are initially integrated at visual perceptual stages of face processing

Abstract

It is frequently assumed that facial identity and facial expression are analysed in functionally and anatomically distinct streams within the core visual face processing system. To investigate whether expression and identity interact during the visual processing of faces, we employed a sequential matching procedure where participants compared either the identity or the expression of two successively presented faces, and ignored the other irrelevant dimension. Repetitions versus changes of facial identity and expression were varied independently across trials, and event-related potentials (ERPs) were recorded during task performance. Irrelevant facial identity and irrelevant expression both interfered with performance in the expression and identity matching tasks. These symmetrical interference effects show that neither identity nor expression can be selectively ignored during face matching, and suggest that they are not processed independently. N250r components to identity repetitions that reflect identity matching mechanisms in face-selective visual cortex were delayed and attenuated when there was an expression change, demonstrating that facial expression interferes with visual identity matching.These findings provide new evidence for interactions between facial identity and expression within the core visual processing system, and question the hypothesis that these two attributes are processed independently.

Facial identity and emotional expression as predictors during economic decisions

“…In this vein, our results suggest that the construction of the perceptual representation of faces when their identity is being used as a relevant cue to anticipate win as the most likely future outcome is influenced by the ignored emotional valence. This supports the interactive nature of identity and facial emotional information from an early stage of processing (Eimer, 2011;Fisher et al, 2016;Hinojosa et al, 2015;Jeffreys, 1989;Joyce & Rossion, 2005;Pitcher et al, 2007;Rossion et al, 2000;Vuilleumier & Pourtois, 2007).…”

“…Smiling faces increased the degree of perceived familiarity for both famous and unknown identities (Baudouin et al, 2000). Also, Fisher, Towler and Eimer (2016) recently observed, during a sequential matching procedure, that identity and emotional expression influence each other at the stage where perceptual face representations are matched with stored memories. In this line, Ganel and Goshen-Gottstein (2004) proposed a structural-reference hypothesis, in which the structure of a face (i.e., its identity) can be employed as a reference image to guide the decoding of idiosyncratic emotional expressions.…”

“…Such emotional information is automatically integrated in the construction of facial identity and also later on when a response must be selected. This interference provides strong evidence in favor of the interdependence between changeable and invariant facial properties from early perceptual processing to response selection stages (Baudouin et al, 2000;Fisher et al, 2016;Ganel and Goshen-Gottstein, 2004;Schweinberger & Soukup, 1998; see also Todorov et al, 2008). As far as emotional expression is a useful tool for predicting outcomes and also a key source of information for social judgments based on faces, the relevance of emotion may be strengthened in social scenarios where personal identity is used as a cue to guide decisions (see for example Ganel and Goshen-Gottstein, 2004;Todorov et al, 2005Todorov et al, , 2008.…”

“…We also expected to observe an interaction between emotion and identity on the N170 potential reflecting incongruence, given its sensitivity to top-down control (Aranda, Madrid, Tudela, & Ruz, 2010;Tortosa, Lupiáñez, et al, 2013;Zhu et al, 2010; although see Fisher et al, 2016). We hypothesized that the N2 potential would be heightened for stimuli with a negative outcome prediction (e.g.…”

“…It is also usually larger for congruent than incongruent trials (Neuhaus et al, 2010;ValleInclán, 1996). In summary, although previous studies have explored how identity and emotion are decoded, these have mostly employed simple categorization tasks (Atkinson, Tipples, Burt, & Young, 2005;Fisher et al, 2016;Schweinberger & Soukup, 1998) or paradigms where only identity or emotion were the relevant elements for predicting outcomes (Alguacil, Tudela, & Ruz, 2015;Ruz et al, 2013;Ruz & Tudela, 2011;Tortosa, Lupiáñez, et al, 2013). Furthermore, most studies to date have employed paradigms fully devoid of social context, which represents a drawback given the innate social nature of emotional phenomena (e.g., Parkinson, 1996).…”