Capturing Complex Spatio-temporal Relations among Facial Muscles for Facial Expression Recognition

Wang, Ziheng; Wang, Shangfei; Ji, Qiang

doi:10.1109/cvpr.2013.439

Cited by 135 publications

(90 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Approach Accuracy (%) ADL [40] 47.78 HMM [41] 51.5 ITBN [41] 59.7 CPL [42] 49.36 CSPL [42] 73.53 STM-ExpLet [16] 75.12 Proposed 78.83 Table 4, respectively.…”

Section: Approachmentioning

confidence: 99%

Natural Facial and Head Behavior Recognition using Dictionary of Motion Primitives

Shi

Ukita

Yang

2017

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYThis paper proposes a natural facial and head behavior recognition method using hybrid dynamical systems. Most existing facial and head behavior recognition methods focus on analyzing deliberately displayed prototypical emotion patterns rather than complex and spontaneous facial and head behaviors in natural conversation environments. We first capture spatio-temporal features on important facial parts via dense feature extraction. Next, we cluster the spatio-temporal features using hybrid dynamical systems, and construct a dictionary of motion primitives to cover all possible elemental motion dynamics accounting for facial and head behaviors. With this dictionary, the facial and head behavior can be interpreted into a distribution of motion primitives. This interpretation is robust against different rhythms of dynamic patterns in complex and spontaneous facial and head behaviors. We evaluate the proposed approach under natural tele-communication scenarios, and achieve promising results. Furthermore, the proposed method also performs favorably against the state-ofthe-art methods on three benchmark databases.

show abstract

“…Approach Accuracy (%) ADL [40] 47.78 HMM [41] 51.5 ITBN [41] 59.7 CPL [42] 49.36 CSPL [42] 73.53 STM-ExpLet [16] 75.12 Proposed 78.83 Table 4, respectively.…”

Section: Approachmentioning

confidence: 99%

Natural Facial and Head Behavior Recognition using Dictionary of Motion Primitives

Shi

Ukita

Yang

2017

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

show abstract

“…Some approaches utilized the texture dynamics for the facial expression recognition [20][21][22]. Many other approaches exploit also the facial point dynamics to recognize the corresponding expression [23][24][25][26][27]. Obviously, these corresponding approaches work with image sequences starting usually with neutral expression.…”

Section: Introductionmentioning

confidence: 99%

Frame-Based Facial Expression Recognition Using Geometrical Features

Anwar

Al-Hamadi

Niese

et al. 2014

Advances in Human-Computer Interaction

View full text Add to dashboard Cite

To improve the human-computer interaction (HCI) to be as good as human-human interaction, building an efficient approach for human emotion recognition is required. These emotions could be fused from several modalities such as facial expression, hand gesture, acoustic data, and biophysiological data. In this paper, we address the frame-based perception of the universal human facial expressions (happiness, surprise, anger, disgust, fear, and sadness), with the help of several geometrical features. Unlike many other geometry-based approaches, the frame-based method does not rely on prior knowledge of a person-specific neutral expression; this knowledge is gained through human intervention and not available in real scenarios. Additionally, we provide a method to investigate the performance of the geometry-based approaches under various facial point localization errors. From an evaluation on two public benchmark datasets, we have found that using eight facial points, we can achieve the state-of-the-art recognition rate. However, this state-of-the-art geometry-based approach exploits features derived from 68 facial points and requires prior knowledge of the person-specific neutral expression. The expression recognition rate using geometrical features is adversely affected by the errors in the facial point localization, especially for the expressions with subtle facial deformations.

show abstract

“…Naive-Bayes classifiers and Gaussian tree-augmented naive Bayes (TAN) classifiers were also used to learn dependencies among different facial motion features. In a series of papers, Qiang and his colleagues (Tong et al, 2007(Tong et al, , 2010Li and Ji, 2013;Wang et al, 2013) used dynamic Bayesian networks to detect AUs. In each of these cases, the goal was improved AUs or facial expressions detection [for a more complete review, please see Zeng et al (2012) and Sariyanidi et al (2015)].…”

mentioning

confidence: 99%

Automatic Measurement of Head and Facial Movement for Analysis and Detection of Infants’ Positive and Negative Affect

et al. 2015

View full text Add to dashboard Cite

Previous work in automatic affect analysis (AAA) has emphasized static expressions to the neglect of the dynamics of facial movement and considered head movement only a nuisance variable to control. We investigated whether the dynamics of head and facial movements apart from specific facial expressions communicate affect in infants, an under-studied population in AAA. Age-appropriate tasks were used to elicit positive and negative affect in 31 ethnically diverse infants. 3D head and facial movements were tracked from 2D video. Head angles in the horizontal (pitch), vertical (yaw), and lateral (roll) directions were used to measure head movement; and the 3D coordinates of 49 facial points to measure facial movements. Strong effects were found for both head and facial movements. Angular velocity and angular acceleration of head pitch, yaw, and roll were higher during negative relative to positive affect. Amplitude, velocity, and acceleration of facial movement were higher as well during negative relative to positive affect. A linear discriminant analysis using head and facial movement achieved a mean classification rate of positive and negative affect equal to 65% (Kappa = 0.30). Head and facial movements individually and in combination were also strongly related to observer ratings of affect intensity. Our results suggest that the dynamics of head and facial movements communicate affect at ages as young as 13 months. These interdisciplinary findings from behavioral science and computer vision deepen our understanding of communication of affect and provide a basis for studying individual differences in emotion in socio-emotional development.

show abstract

Capturing Complex Spatio-temporal Relations among Facial Muscles for Facial Expression Recognition

Cited by 135 publications

References 13 publications

Natural Facial and Head Behavior Recognition using Dictionary of Motion Primitives

Natural Facial and Head Behavior Recognition using Dictionary of Motion Primitives

Frame-Based Facial Expression Recognition Using Geometrical Features

Automatic Measurement of Head and Facial Movement for Analysis and Detection of Infants’ Positive and Negative Affect

Contact Info

Product

Resources

About