Comparing a novel model based on the transferable belief model with humans during the recognition of partially occluded facial expressions

Abstract

Humans recognize basic facial expressions effortlessly. Yet, despite a considerable amount of research, this task remains elusive for computer vision systems. Here, we compared the behavior of one of the best computer models of facial expression recognition (Z. Hammal, L. Couvreur, A. Caplier, & M. Rombaut, 2007) with the behavior of human observers during the M. Smith, G. Cottrell, F. Gosselin, and P. G. Schyns (2005) facial expression recognition task performed on stimuli randomly sampled using Gaussian apertures. The model--which we had to significantly modify in order to give the ability to deal with partially occluded stimuli--classifies the six basic facial expressions (Happiness, Fear, Sadness, Surprise, Anger, and Disgust) plus Neutral from static images based on the permanent facial feature deformations and the Transferable Belief Model (TBM). Three simulations demonstrated the suitability of the TBM-based model to deal with partially occluded facial parts and revealed the differences between the facial information used by humans and by the model. This opens promising perspectives for the future development of the model.

A computational shape-based model of anger and sadness justifies a configural representation of faces

“…Their results reinforce the claim that shape-based representations are important in face analysis. In another recent paper, Hammal et al (2009) show that similar descriptions of faces can classify expressions of emotion similarly to human subjects (even when the percept is partially occluded). In the present paper, we have shown how such a shape-based representation justifies the emergence of these configural cues in the recognition of anger and sadness.…”

Random Gabor based templates for facial expression recognition in images with facial occlusion

“…Existing FER studies on handling occlusion try to reconstruct the occluded geometric or texture features based on the configuration and visual properties of human face. Techniques for reconstructing geometry include principal component analysis (PCA) [2], the improved Kanade-Lucas tracker [3], [4], Bayesian tracker [5], and the transferable belief model [6], and those for reconstructing texture rely on robust PCA [7], [8]. These systems depend heavily on accurate detection of geometric points and occlusion regions.…”

Facial Expression Analysis under Partial Occlusion

“…A unique feature of statistical model approaches lies in the capacity of robustly inferring facial features in a current frame based on facial information in neighboring frames, even when the current frame contains a partially occluded face or is a completely missing frame. Hammal et al [2009] exploited the usefulness of facial point deformations with a modified Transferable Belief Model (TBM) for recognizing facial expressions from images with partial occlusion. Five distances from the contours of the mouth, eyes, and eyebrows were normalized and mapped to symbolic states.…”

“…The behaviors of human observers are different from TBM based models and the human tends to use "suboptimal" features for FEA under occlusion. Rather than using distances between facial components in static images [Hammal, et al 2009], Miyakoshi and Kato [2011] improved this by using movement magnitudes of 14 points from a neutral face to an emotional face in static images. A Bayesian network classifier was employed to learn the dependencies between target facial expressions and facial features without involving a process of filling in the facial gap due to occlusion.…”