Removing hand form information specifically impairs emotion recognition for fearful and angry body stimuli

Abstract

Emotion perception research has largely been dominated by work on facial expressions, but emotion is also strongly conveyed from the body. Research exploring emotion recognition from the body tends to refer to ‘the body’ as a whole entity. However, the body is made up of different components (hands, arms, trunk etc.), all of which could be differentially contributing to emotion recognition. We know that the hands can help to convey actions, and in particular are important for social communication through gestures, but we currently do not know to what extent the hands influence emotion recognition from the body. Here, 93 adults viewed static emotional body stimuli with either the hands, arms, or both components removed and completed a forced-choice emotion recognition task. Removing the hands significantly reduced recognition accuracy for fear and anger, but made no significant difference to the recognition of happiness and sadness. Removing the arms had no effect on emotion recognition accuracy compared to the full-body stimuli. These results suggest the hands may play a key role in the recognition of emotions from the body.

The representation of bodies in high level visual, prefrontal and inferior parietal cortex varies with explicit vs. implicit expression perception

“…Another possibility to consider is that the observed effects in EBA reflect recognition of the body expression that would only be based on a part of the body like the hands and not on the whole body. Recent evidence shows that the hands are more informative for certain emotions, including anger images used here (Poyo Solanas, Vaessen, and de Gelder 2020a; Kret and de Gelder 2012; Kret et al 2017; Ross and Flack 2020). Concerning representation in the brain (Taylor, Wiggett, and Downing 2007) found that bilateral EBA showed a preference for individual body parts such as the hands and fingers while FBA showed a preference for the whole body.…”

Validation of the bodily expressive action stimulus test among Chinese adults and children

“…The original study selected 19 Western adults to conduct a standardized assessment of the stimulus set, and found that Western adults had a high consistency of assessment, and the accuracy of identifying negative emotions was higher than that of positive emotions (de Gelder & Van den Stock, 2011). Some studies have confirmed the availability of this database (Bannerman et al, ; Hajduk et al, 2019; Ross & Flack, 2019; Watson & de Gelder, 2020).…”

“…Children tended to mistake sadness and anger for fear, perhaps because children around 5 years old cannot use reliable cues to distinguish sadness and fear as adults do (Boone & Cunningham, 2001). The confusion about happiness, anger, and fear can be traced to the confusion of anger and happiness, which may be due to the similarity of selected body postures and expressions, such as the tendency of arms to lean forward (Ross & Flack, 2019).…”

“…Analysis of the proportion of incorrect recognitions of emotion indicated that adults were more likely to confuse positive emotions with negative emotions, which confirmed the findings of previous studies (Atkinson et al, 2004; Dittrich et al, 1996; Gross et al, 2010). The confusion of these two emotions can be explained by the similarity of body postures, such as expressing emotions accompanied by clenched fists and raised arms (Dittrich et al, 1996; Ross & Flack, 2019). These features were found in the pictures we selected.…”

Hand Tracking for Immersive Virtual Reality: Opportunities and Challenges

“…However, the involuntary movements of hands during natural communication appear to play a significant role not just for the listener, but also the communicator (the latter to such an extent that conversations between two congenitally blind individuals contain as many gestures as conversations between sighted individuals). Indeed, recent research has shown that individuals are impaired in recognizing a number of key emotions in the images of bodies which have the hands removed (Ross and Flack, 2020), highlighting how important hand form information is in communicative experiences. The value of manual gestures for communication in virtual environments is compounded given that veridical real-time face tracking and visualisation is technically very difficult due to the extremely high temporal and spatial resolution required to detect and track microexpressions.…”