Naturalistic Emotion Decoding From Facial Action Sets

Abstract

Researchers have theoretically proposed that humans decode other individuals' emotions or elementary cognitive appraisals from particular sets of facial action units (AUs). However, only a few empirical studies have systematically tested the relationships between the decoding of emotions/appraisals and sets of AUs, and the results are mixed. Furthermore, the previous studies relied on facial expressions of actors and no study used spontaneous and dynamic facial expressions in naturalistic settings. We investigated this issue using video recordings of facial expressions filmed unobtrusively in a real-life emotional situation, specifically loss of luggage at an airport. The AUs observed in the videos were annotated using the Facial Action Coding System. Male participants (n = 98) were asked to decode emotions (e.g., anger) and appraisals (e.g., suddenness) from facial expressions. We explored the relationships between the emotion/appraisal decoding and AUs using stepwise multiple regression analyses. The results revealed that all the rated emotions and appraisals were associated with sets of AUs. The profiles of regression equations showed AUs both consistent and inconsistent with those in theoretical proposals. The results suggest that (1) the decoding of emotions and appraisals in facial expressions is implemented by the perception of set of AUs, and (2) the profiles of such AU sets could be different from previous theories.

Contextual effects on smile perception and recognition memory

“…Studying the impact of context on emotion perception is essential, as presenting participants with face stimuli lacking contextual information creates an experimental setting that has low ecological validity. In real life, observers process contextual information to accurately interpret the meaning of emotion displays, especially given that displays observed in naturalistic emotional situations are nuanced and include subtleties (Hyniewska et al, 2019) which make them challenging to univocally categorize per se (Hess et al, 1997;Sato et al, 2019).…”

Semantics Based on the Physical Characteristics of Facial Expressions Used to Produce Japanese Vowels

“…Indeed, Durán et al showed in their meta-analysis on the coherence between facial expressions and emotional experiences that prototypical whole facial patterns were not coherent with the emotional experiences of the expressers, whereas there was a small coherence between their emotional experiences and individual facial components [ 19 ]. Moreover, Hyniewska et al illustrated how each facial component could be associated with the perceivers’ specific decoded emotions [ 20 ]. Furthermore, Reschke et al showed that physical changes of the mouth expressing disgust induced different emotional perceptions [ 21 ]; the disgust expression with an open mouth, compared to the disgust expression with a closed mouth, can lead to different inferences according to the type of contextual information available, such as posture-scene stimuli.…”

Dynamic Facial Expression of Emotion and Observer Inference

“…The results showed that “felt,” but not “false” smiles [as defined by Ekman and Friesen ( 1982 )] correlated strongly positively with a “in good humor” scale in agent ratings and both types of judges' ratings, but only weakly so with self-ratings. The video material collected by Scherer and Ceschi in this field study was used by Hyniewska et al ( 2018 ) to study the emotion antecedent appraisals (see Scherer, 2001 ) and the resulting emotions of the voyagers claiming lost baggage inferred by judges on the basis of the facial expressions. The videos were annotated with the FACS system and stepwise regression was used to identify the AUs predicting specific inferences.…”