Rapid perceptual integration of facial expression and emotional body language

Abstract

In our natural world, a face is usually encountered not as an isolated object but as an integrated part of a whole body. The face and the body both normally contribute in conveying the emotional state of the individual. Here we show that observers judging a facial expression are strongly influenced by emotional body language. Photographs of fearful and angry faces and bodies were used to create face-body compound images, with either matched or mismatched emotional expressions. When face and body convey conflicting emotional information, judgment of facial expression is hampered and becomes biased toward the emotion expressed by the body. Electrical brain activity was recorded from the scalp while subjects attended to the face and judged its emotional expression. An enhancement of the occipital P1 component as early as 115 ms after presentation onset points to the existence of a rapid neural mechanism sensitive to the degree of agreement between simultaneously presented facial and bodily emotional expressions, even when the latter are unattended.emotion communication ͉ event-related potentials ͉ visual perception T he face and the body both normally contribute in conveying the emotional state of the individual. Darwin (1) was the first to describe in detail the specific facial and bodily expressions associated with emotions in animals and humans and regarded these expressions as part of emotion-specific adaptive actions. From this vantage point, face and body are part of an integrated whole. Indeed, in our natural world, faces are usually encountered not as isolated objects but as an integrated part of a whole body. Rapid detection of inconsistencies between them is beneficial when rapid adaptive action is required from the observer. To date, there has been no systematic investigation into how facial expressions and emotional body language interact in human observers, and the underlying neural mechanisms are unknown.Here, we investigate the inf luence of emotional body language on the perception of facial expression. We collected behavioral data and simultaneously measured electrical eventrelated potentials (ERP) from the scalp to explore the time course of neuronal processing with a resolution in the order of milliseconds. We predicted that recognition of facial expressions is inf luenced by concurrently presented emotional body language, and that affective information from the face and the body start to interact rapidly. In view of the important adaptive function of perceiving emotional states and previous findings about rapid recognition of emotional signals, we deemed it unlikely that integrated perception of face-body images results from relatively late and slow semantic processes. We hypothesize that the integration of affective information from a facial expression and the accompanying emotional body language is a mandatory automatic process occurring early in the processing stream, which does not require selective attention, thorough visual analysis of individual features, or conscious deliberate evaluatio...

Faces, Bodies, Social Vision as Agent Vision, and Social Consciousness

“…The N170 is a well-known negative ERP-component, peaking around 170 ms at occipito-temporal sites, and is often linked to the stage of the structural encoding of faces (Bentin, Allison, Puce, Perez, & et al, 1996;Bruce & Young, 1986;Eimer, 2000b). Yet this component is also elicited by images of whole bodies (Gliga & Dehaene-Lambertz, 2005;Meeren, van Heijnsbergen, & de Gelder, 2005;Stekelenburg & de Gelder, 2004;Thierry et al, 2006). Most importantly, the electrophysiological inversion effect (Eimer, 2000a;Itier & Taylor, 2002;Watanabe, Kakigi, & Puce, 2003), consisting of a delayed and enhanced N170 deflection for inverted stimulus presentation, was elicited by both faces and bodies in contrast to control objects (Stekelenburg & de Gelder, 2004).…”

“…In a first investigation of the time course, EEG was Faces, Bodies, Social Vision as Agent Vision, and Social Consciousness -Oxford Scholarship Pagina 12 di 35 http://www.oxfordscholarship.com/view/10.1093/acprof:oso/9780195333176.001.0001/acprof-9780195333176-chapter-4?print used (Meeren et al, 2005). The combined processing of facial and bodily expressions revealed behavioral and rapid electrophysiological effects of emotional congruency.…”

“…The A in column heading denotes main effects for amplitudes. Modified from Meeren et al, 2005. strictly cortical taxonomies to processes in midbrain and in phylogenetically older subcortical structures. This is the area in which the notion of a brain equipped with social vision may be best implemented.…”

Why bodies? Twelve reasons for including bodily expressions in affective neuroscience

“…Two behavioural studies directly investigated how our recognition of facial expressions is influenced by accompanying wholebody expressions (Meeren et al 2005; Van den Stock et al 2007). Meeren et al (2005) combined angry and fearful facial expressions with angry and fearful whole-body expressions to create both congruent (a fearful face on a fearful body and an angry face on an angry body) and incongruent (a fearful face on an angry body and an angry face on a fearful body) realistic looking compound stimuli. These were briefly (200 ms) presented one by one while the participants were instructed to categorize the emotion expressed by the face and ignore the body.…”

Using Intelligent 3D Animated Character as the Interface for Interactive Digital TV System

“…However, if a virtual character on the screen is not large, facial expression may be difficult to tell. As pointed out by Meeren [6], body expressions have higher priorities than facial expressions when perceived by a viewer. Therefore, in this work, we use full-body gestures to express emotion when delivering a message.…”