DISFA: A Spontaneous Facial Action Intensity Database

Mavadati, S. Mohammad; Mahoor, Mohammad H.; Bartlett, Katharine T.; Trinh, Phan Trong; Cohn, Jeffrey F.

doi:10.1109/t-affc.2013.4

Cited by 637 publications

(392 citation statements)

References 53 publications

Supporting

Mentioning

390

Contrasting

Unclassified

Order By: Relevance

“…Although deep learning algorithms have been shown to produce state of the art performance on object recognition tasks, there has been considerably less work on using deep learning techniques in action recognition, facial expression recognition, and in particular facial AU recognition. With the increasing availability of large databases for AU recognition [16,23,29], it would be interesting to see if deep learning algorithms can give a similar leap in performance in the field of facial expression/AU recognition.…”

Section: Introductionmentioning

confidence: 99%

Deep learning the dynamic appearance and shape of facial action units

Jaiswal

Valstar

2016

2016 IEEE Winter Conference on Applications of Computer Vision (WACV)

138

View full text Add to dashboard Cite

Spontaneous facial expression recognition under uncontrolled conditions is a hard task. It depends on multiple factors including shape, appearance and dynamics of the facial features, all of which are adversely affected by environmental noise and low intensity signals typical of such conditions. In this work, we present a novel approach to Facial Action Unit detection using a combination of Convolutional and Bi-directional Long Short-Term Memory Neural Networks (CNN-BLSTM), which jointly learns shape, appearance and dynamics in a deep learning manner. In addition, we introduce a novel way to encode shape features using binary image masks computed from the locations of facial landmarks. We show that the combination of dynamic CNN features and Bi-directional Long Short-Term Memory excels at modelling the temporal information. We thoroughly evaluate the contributions of each component in our system and show that it achieves state-of-the-art performance on the FERA-2015 Challenge dataset.

show abstract

Section: Introductionmentioning

confidence: 99%

Deep learning the dynamic appearance and shape of facial action units

Jaiswal

Valstar

2016

2016 IEEE Winter Conference on Applications of Computer Vision (WACV)

138

View full text Add to dashboard Cite

show abstract

“…Evaluation of the proposed approach is performed on the Denver Intensity of Spontaneous Facial Actions (DISFA) dataset [26], the recently published dataset of naturalistic facial AUs that are FACS coded in terms of their intensity using the ordinal scores: 0 (not present) to 5 (maximum intensity). This dataset consists of video recordings of Fig.…”

Section: Methodsmentioning

confidence: 99%

Personalized Modeling of Facial Action Unit Intensity

Yang

Rudovic

Pavlović

et al. 2014

Advances in Visual Computing

View full text Add to dashboard Cite

Abstract. Facial expressions depend greatly on facial morphology and expressiveness of the observed person. Recent studies have shown great improvement of the personalized over non-personalized models in variety of facial expression related tasks, such as face and emotion recognition. However, in the context of facial action unit (AU) intensity estimation, personalized modeling has been scarcely investigated. In this paper, we propose a two-step approach for personalized modeling of facial AU intensity from spontaneously displayed facial expressions. In the first step, we perform facial feature decomposition using the proposed matrix decomposition algorithm that separates the person's identity from facial expression. These two are then jointly modeled using the framework of Conditional Ordinal Random Fields, resulting in a personalized model for intensity estimation of AUs. Our experimental results show that the proposed personalized model largely outperforms non-personalized models for intensity estimation of AUs.

show abstract

“…Based on the modeling approach, these can be divided into static methods (Mahoor et al 2009, Mavadati et al 2013, Savrana et al 2012, Kaltwang et al 2012, Jeni et al 2013) and dynamic methods (Rudovic et al 2013b). The static methods can further be divided into classification-based methods (e.g., (Mahoor et al 2009, Mavadati et al 2013) and regression-based (e.g, (Savrana et al 2012, Kaltwang et al 2012, Jeni et al 2013). The static classification-based methods (Mahoor et al 2009, Mavadati et al 2013) perform multi-class classification of the intensity of AUs using the SVM classifier.…”

Section: Intensity Estimation Of Facial Expressionsmentioning

confidence: 99%

“…The static methods can further be divided into classification-based methods (e.g., (Mahoor et al 2009, Mavadati et al 2013) and regression-based (e.g, (Savrana et al 2012, Kaltwang et al 2012, Jeni et al 2013). The static classification-based methods (Mahoor et al 2009, Mavadati et al 2013) perform multi-class classification of the intensity of AUs using the SVM classifier. For example, (Mahoor et al 2009) performed the intensity estimation of AU6 (cheek raiser) and AU12 (lip corner puller) from facial images of infants.…”

Section: Intensity Estimation Of Facial Expressionsmentioning

confidence: 99%