Looking beyond the static face in emotion recognition: The informative case of interest

Abstract

A new wave of studies on emotion recognition encourages researchers to look beyond the face as the sole source of pertinent information. One study has proposed that, while there is overwhelming evidence that negative emotions may be differentiated in static facial expressions, postural information is needed to differentiate positive emotions such as pride, love and amusement. We review the most influential of these recent studies before providing some supporting evidence in an exploratory study of a fourth positive emotion -interest -that adds to the recent calls to move beyond static facial expressions of emotion when investigating how we determine how others are feeling and how they are likely to act next.

Behavioral and Physiological Responses to Visual Interest and Appraisals: Multimodal Analysis and Automatic Recognition

“…Curiosity has a similar pattern. Our results confirm other recent studies that found interest correlated to smiling behavior and in general being part of the category of positive emotions (Mortillaro et al, 2011;Campos et al, 2013;Dukes et al, 2017). An alternative explanation would be that we did not have extremely unpleasant stimuli that could have elicited interest but not a smiling behavior.…”

“…Mortillaro et al (2011) suggested that the specific movements able to differentiate among positive emotions should be looked for in the unfolding of the expression, in its dynamics, and not in a static facial configuration. Dukes et al (2017) explored this possibility and found that indeed adding dynamic information significantly increased the accuracy with which the facial expressions of interest were recognized by others, and this effect was larger for interest than for other emotions. Importantly, when viewers could see only a static facial expression, the accuracy was only 29% (a figure that should be compared to an accuracy of 68% on average for the other emotions).…”

“…More precisely, the pattern includes arms resting at side, trunk leaning forward, and asymmetrical one-arm actions. Dukes et al (2017) compared the nonverbal expression of interest in the body and the accuracy with which the emotion was recognized with that of five basic emotions. The results showed that, when the observer could see the body in addition to the facial expression, interest was as well recognized as the other emotions.…”

“…Researchers investigated other expressive modalities beyond facial expression, in particular the bodily movements that accompany the experience of interest. The body may indeed play a special role in communicating specific positive emotions (Tracy and Robins, 2004;Gonzaga et al, 2006;Dukes et al, 2017;Mortillaro and Dukes, 2018). A survey on body expression and perception identified body postures and head pose to be related to expression of interest and boredom (Kleinsmith and Bianchi-Berthouze, 2013).…”

“…Most researchers agree that interest is an emotion and therefore is characterized by a specific cognitive appraisal structure, a consistent set of bodily expressions, and an action tendency of exploration (Izard, 2009;Mortillaro et al, 2011;Campos et al, 2013;Dukes et al, 2017). Appraisals are cognitive evaluations that occur with or without awareness -when a person faces an event or object.…”