Attributing Emotion to Static Body Postures: Recognition Accuracy, Confusions, and Viewpoint Dependence

Coulson, Mark

doi:10.1023/b:jonb.0000023655.25550.be

Cited by 460 publications

(320 citation statements)

References 45 publications

Supporting

Mentioning

286

Contrasting

Unclassified

Order By: Relevance

“…Our results suggest that the dynamic property of repetitiveness is an additional essential aspect of emotion expression. However, the association between joy-like emotions and upward arm movement has been repeatedly reported in emotion expression literature (Boone & Cunningham, 1998, 2001Coulson, 2004;de Meijer, 1989;Wallbott, 1998).…”

Section: Theoretical Evaluation Of the Obtained Resultsmentioning

confidence: 98%

Emotion expression in body action and posture.

Dael¹,

Mortillaro²,

Scherer³

2012

Emotion

357

282

View full text Add to dashboard Cite

Emotion communication research strongly focuses on the face and voice as expressive modalities, leaving the rest of the body relatively understudied. Contrary to the early assumption that body movement only indicates emotional intensity, recent studies have shown that body movement and posture also conveys emotion specific information. However, a deeper understanding of the underlying mechanisms is hampered by a lack of production studies informed by a theoretical framework. In this research we adopted the Body Action and Posture (BAP) coding system to examine the types and patterns of body movement that are employed by 10 professional actors to portray a set of 12 emotions. We investigated to what extent these expression patterns support explicit or implicit predictions from basic emotion theory, bidimensional theory, and componential appraisal theory. The overall results showed partial support for the different theoretical approaches. They revealed that several patterns of body movement systematically occur in portrayals of specific emotions, allowing emotion differentiation. Although a few emotions were prototypically expressed by one particular pattern, most emotions were variably expressed by multiple patterns, many of which can be explained as reflecting functional components of emotion such as modes of appraisal and action readiness. It is concluded that further work in this largely underdeveloped area should be guided by an appropriate theoretical framework to allow a more systematic design of experiments and clear hypothesis testing.

show abstract

Section: Theoretical Evaluation Of the Obtained Resultsmentioning

confidence: 98%

Emotion expression in body action and posture.

Dael¹,

Mortillaro²,

Scherer³

2012

Emotion

357

282

View full text Add to dashboard Cite

show abstract

“…To the best of our knowledge, no other study has addressed the issue of automated emotion recognition based on the face, body gesture and speech modalities in order to attempt to infer 8 emotions. Although several research works investigated the importance of gesture in emotion recognition systems (see, for example, [56], [10], [11], [12]) and a few studies have been successful in pairing of body gesture and facial expression for recognizing affective expressions (see, for example, Gunes and Piccardi [8], el Kaliouby and Robinson 2005 [36], Balomenos et al [57]), the use of body gesture in this work is novel in the sense that no other study has added this modality when trying to improve the performance of an emotion detector based on facial expression and speech analysis.…”

Section: Discussionmentioning

confidence: 99%

Multimodal emotion recognition in speech-based interaction using facial expression, body gesture and acoustic analysis

Kessous¹,

Castellano

2009

J Multimodal User Interfaces

204

View full text Add to dashboard Cite

In this paper a study on multimodal automatic emotion recognition during a speech-based interaction is presented. A database was constructed consisting of people pronouncing a sentence in a scenario where they interacted with an agent using speech. Ten people pronounced a sentence corresponding to a command while making 8 different emotional expressions. Gender was equally represented, with speakers of several different native languages including French, German, Greek and Italian. Facial expression, gesture and acoustic analysis of speech were used to extract features relevant to emotion. For the automatic classification of unimodal data, bimodal data and multimodal data, a system based on a Bayesian classifier was used. After performing an automatic classification of each modality, the different modalities were combined using a multimodal approach. Fusion of the modalities at the feature level (before running the classifier) and at the results level (combining results from classifier from each modality) were compared. Fusing the multimodal data resulted in a large increase in the recognition rates in comparison to the unimodal systems: the multimodal approach increased the recognition rate by more than 10% when compared to the most successful unimodal system. Bimodal emotion recognition based on all combinations of the modalities (i.e., 'face-gesture', 'facespeech' and 'gesture-speech') was also investigated. The results show that the best pairing is 'gesture-speech'. Using all three modalities resulted in a 3.3% classification improvement over the best bimodal results.

show abstract

“…Research on emotion recognition has been dominated by studies using photographs of facial expressions, yet movement of the body or its parts is also revealing of other people's emotions. Using standard recognition tasks, such as those requiring forced choices from a list of emotion labels or judgments of emotional intensity, several studies have demonstrated that, even in the absence of facial and vocal cues, humans are adept at identifying basic emotions i signaled by static body postures (e.g., Atkinson, Dittrich, Gemmell, & Young, 2004;Coulson, 2004), arm movement (Pollick, Paterson, Bruderlin, & Sanford, 2001), and whole body movement (e.g., Atkinson et al, 2004;de Meijer, 1989;Dittrich, Troscianko, Lea, & Morgan, 1996). On what visual information is this ability based?…”

Section: Evidence For Distinct Contributions Of Form and Motion Informentioning

confidence: 99%

Evidence for distinct contributions of form and motion information to the recognition of emotions from body gestures

2007

View full text Add to dashboard Cite

Publisher's copyright statement:Additional information: Use policyThe full-text may be used and/or reproduced, and given to third parties in any format or medium, without prior permission or charge, for personal research or study, educational, or not-for-prot purposes provided that:• a full bibliographic reference is made to the original source • a link is made to the metadata record in DRO • the full-text is not changed in any way The full-text must not be sold in any format or medium without the formal permission of the copyright holders.Please consult the full DRO policy for further details.

show abstract

Attributing Emotion to Static Body Postures: Recognition Accuracy, Confusions, and Viewpoint Dependence

Cited by 460 publications

References 45 publications

Emotion expression in body action and posture.

Emotion expression in body action and posture.

Multimodal emotion recognition in speech-based interaction using facial expression, body gesture and acoustic analysis

Evidence for distinct contributions of form and motion information to the recognition of emotions from body gestures

Contact Info

Product

Resources

About