Featural processing in recognition of emotional facial expressions

Abstract

The present study aimed to clarify the role played by the eye/brow and mouth areas in the recognition of the six basic emotions. In Experiment 1, accuracy was examined while participants viewed partial and full facial expressions; in Experiment 2, participants viewed full facial expressions while their eye movements were recorded. Recognition rates were consistent with previous research: happiness was highest and fear was lowest. The mouth and eye/brow areas were not equally important for the recognition of all emotions. More precisely, while the mouth was revealed to be important in the recognition of happiness and the eye/brow area of sadness, results are not as consistent for the other emotions. In Experiment 2, consistent with previous studies, the eyes/brows were fixated for longer periods than the mouth for all emotions. Again, variations occurred as a function of the emotions, the mouth having an important role in happiness and the eyes/brows in sadness. The general pattern of results for the other four emotions was inconsistent between the experiments as well as across different measures. The complexity of the results suggests that the recognition process of emotional facial expressions cannot be reduced to a simple feature processing or holistic processing for all emotions.

Dynamic facial expressions are processed holistically, but not more holistically than static facial expressions

“…Further, our results suggest that the degree of holistic processing depends on the expression to be identified and also on whether expressions are static or dynamic. This is consistent with the idea that the perception of facial expression varies across emotion and is not exclusively holistic or featural but likely to involve some combination of both processes (Beaudry et al, 2014;Tanaka et al, 2012). Overall, the magnitude of holistic processing for dynamic expressions appeared to be either at a similar level to or reduced when compared to static expressions.…”

“…In line with the idea that the contribution of holistic and featural information to expression recognition varies with individual emotions (Beaudry, et al, 2014;Bombari et al, 2013;Calvo & Nummennmaa, 2008), comparison of the magnitude of the composite effect across expressions revealed an inconsistent pattern, which depended on whether the expressions were static or dynamic. For expressions that were reliably recognised from the bottom-half of the face (joy, fear and disgust), the composite effect was significantly larger in static than dynamic stimuli.…”

“…In regards to the information that may be used to form visual representations, there is evidence for individual features being necessary and sufficient for the recognition of some expressions, such as an upturned mouth in the recognition of happiness (e.g., Beaudry et al, 2014), whereas other expressions seem to rely upon the processing of multiple features, such as the nose and mouth for disgust (which could be processed sequentially, see Beaudry et al, 2014 or simultaneously but not holistically, see Ellison & Massaro, 1997). There is also evidence that some expressions, such as fear (e.g., Beaudry et al 2014) may be processed more holistically, in which there is very strong perceptual integration across the whole face simultaneously so that the whole perceived is more than the sum of the parts (see detailed definitions in McKone et al, 2013;Rossion, 2013).…”

“…The perception of emotion is undeniably complex (see Beaudry, Roy-Charland, Perron, Cormier, & Tapp, 2014 for recent further discussion), with evidence that facial expression may be coded relative to a norm (Burton, Jeffery, Calder, & Rhodes, 2015;Skinner & Benton, 2010) and may be "embodied" involving both visual processing and simulation of somatosensory brain regions specific to each emotion (Pitcher, Garrido, Walsh, & Duchaine, 2008). In regards to the information that may be used to form visual representations, there is evidence for individual features being necessary and sufficient for the recognition of some expressions, such as an upturned mouth in the recognition of happiness (e.g., Beaudry et al, 2014), whereas other expressions seem to rely upon the processing of multiple features, such as the nose and mouth for disgust (which could be processed sequentially, see Beaudry et al, 2014 or simultaneously but not holistically, see Ellison & Massaro, 1997).…”

CLBPGNN Convoluted Local Binary Pattern based Grouping Neural Network for Face Emotion Recognition

“…Typically, the emotional reactions of human beings is complicated at some times, so it is very difficult to predict the emotions due to the mixture of reactions [7,8]. It includes anger, fear, happy, sadness, contempt, disgust, and surprise.…”