Recognition of 3D facial expression dynamics

Sandbach, Georgia; Zafeiriou, Stefanos; Pantić, Maja; Rueckert, Daniel

doi:10.1016/j.imavis.2012.01.006

Cited by 105 publications

(79 citation statements)

References 44 publications

(71 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In addition, facial expressions have been analyzed [24] and classified [25][26][27] using TS imaging. Commonly, VS imaging has been used for modeling facial expressions, and associated robust facial recognition techniques have been developed [28][29][30]. However, from our understanding, the literature has not developed computational models for stress recognition using both TS and VS imaging together as yet.…”

Section: Introductionmentioning

confidence: 99%

Thermal spatio-temporal data for stress recognition

et al. 2014

View full text Add to dashboard Cite

Stress is a serious concern facing our world today, motivating the development of a better objective understanding through the use of non-intrusive means for stress recognition by reducing restrictions to natural human behavior. As an initial step in computer vision-based stress detection, this paper proposes a temporal thermal spectrum (TS) and visible spectrum (VS) video database ANUStressDB -a major contribution to stress research. The database contains videos of 35 subjects watching stressed and not-stressed film clips validated by the subjects. We present the experiment and the process conducted to acquire videos of subjects' faces while they watched the films for the ANUStressDB. Further, a baseline model based on computing local binary patterns on three orthogonal planes (LBP-TOP) descriptor on VS and TS videos for stress detection is presented. A LBP-TOP-inspired descriptor was used to capture dynamic thermal patterns in histograms (HDTP) which exploited spatio-temporal characteristics in TS videos. Support vector machines were used for our stress detection model. A genetic algorithm was used to select salient facial block divisions for stress classification and to determine whether certain regions of the face of subjects showed better stress patterns. Results showed that a fusion of facial patterns from VS and TS videos produced statistically significantly better stress recognition rates than patterns from VS or TS videos used in isolation. Moreover, the genetic algorithm selection method led to statistically significantly better stress detection rates than classifiers that used all the facial block divisions. In addition, the best stress recognition rate was obtained from HDTP features fused with LBP-TOP features for TS and VS videos using a hybrid of a genetic algorithm and a support vector machine stress detection model. The model produced an accuracy of 86%.

show abstract

Section: Introductionmentioning

confidence: 99%

Thermal spatio-temporal data for stress recognition

et al. 2014

View full text Add to dashboard Cite

show abstract

“…The recent advances of 3D/4D sensors based on two main technologies -stereo-photogrammetry and structured-light -opened the doors to develop approaches for analyzing static (3D) and dynamic (4D) shapes. In particular, targeting the face recognition or expression classification problems, few groups have recently collected new datasets (Sun and Yin, 2008) (Zhang et al, 2014) (Cosker et al, 2011) and have developed first techniques (Sun et al, 2010) (Sandbach et al, 2012) (Ben Amor et al, 2014) for expression recognition from 3D sequences. These approaches have demonstrated the role of 3D dynamic faces analysis to reveal deformations hidden in 2D videos.…”

Section: Prior Workmentioning

confidence: 99%

“…Specifically, in comparing shapes of faces, it is important that similar biological parts are registered to each other, in particular the left and right halves of the face, when studying the face asymmetry. Several methods have been proposed in the literature as discussed above such as the Non-rigid ICP algorithm (Cheng et al, 2015), the Free Form Deformation (FFD) algorithm (Sandbach et al, 2012) and the Thin-plate Spline (TPS) algorithm (Fang et al, 2012). Most of these solutions try to find an optimal registration between two 3D faces, however, their cost functions which minimize the distance between 3D meshes is not a proper metric; it is not even symmetric.…”

Section: Pre-processing Of 3d Framesmentioning

confidence: 99%

Facial Asymmetry Assessment from 3D Shape Sequences: The Clinical Case of Facial Paralysis

Desrosiers

Bennis

Amor

et al. 2016

Proceedings of the 11th Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications

View full text Add to dashboard Cite

Abstract:This paper addresses the problem of quantifying the facial asymmetry from dynamic 3D data. We investigate here the role of 4D (i.e. 3D+time) data to reveal the amount of both static and dynamic asymmetry, in a clinical use-case of facial paralysis. The final goal is to provide tools and solutions to clinicians for facial paralysis assessment and monitoring, which can provide qualitative and quantitative evaluations. To this end, the approach proposed here consider 3D facial sequences and adopt a recently-developed Riemannian approach for facial deformation analysis. After a preprocessing step, each frame of a given 3D sequence is approximated by an indexed collection of elastic radial curves. Riemannian shape analysis of obtained curves and their symmetrical counterparts, both elements of the same shape space, give rise to a feature vector, called Dense Scalar Fields (DSFs). The use of these DSFs reveals the amount of bilateral asymmetry of the face, when conveying expressions. That is, given a 3D frame, it is first reflected with respect to the YZ-plane, then compared to the obtained reflection using the DSFs. To exemplify the use of the proposed approach, a new dataset have been collected (of patients) before and after injecting Botulinum Toxin (BT) in related facial muscles. Experimental results obtained on this dataset show that the proposed approach allows clinicians to evaluate the facial asymmetry before and after the medical treatment.

show abstract

“…Many researchers have presented methods for automatic recognition of emotions from videos. For recognizing emotions using BU4DFE, a method was proposed using Iterative Closest Point (ICP), Free Form Deformation (FFD), vector projections and Hidden Markov Model [1]. An emotional avatar image concept using FERA 2011 database has been developed in [2].…”

Section: Introductionmentioning

confidence: 99%

Geometrical Approach for Emotion Recognition from Facial Expressions Using 4D Videos and Analysis on Feature-Classifier Combination

Palaniswamy¹,

Tripathi²

2017

IJIES

View full text Add to dashboard Cite

Emotion recognition from facial expressions using videos is important in human computer communication where the continuous changes in face movements need to be recognized efficiently. In this paper, a method using the geometrical based approach for feature extraction and recognition of six basic emotions has been proposed which is named as GAFCI (Geometrical Approach for Feature Classifier Identification). Various classifiers, Support Vector Machine (SVM), Random Forest, Naïve Bayes and Neural Networks are used for classification, and the performances of all the chosen classifiers are compared. Out of the 83 feature points provided in the BU4DFE database, optimum feature points are identified by experimenting with several sets of feature points. Suitable "feature-classifier" combination has been obtained by varying the number of feature points, classifier parameters, and training and test samples. A detailed analysis on the feature points and classifiers has been performed to learn the relationship between distance parameters and classification of emotions. The results are compared with literature and found to be encouraging.

show abstract

Recognition of 3D facial expression dynamics

Cited by 105 publications

References 44 publications

Thermal spatio-temporal data for stress recognition

Thermal spatio-temporal data for stress recognition

Facial Asymmetry Assessment from 3D Shape Sequences: The Clinical Case of Facial Paralysis

Geometrical Approach for Emotion Recognition from Facial Expressions Using 4D Videos and Analysis on Feature-Classifier Combination

Contact Info

Product

Resources

About