Deep spatial-temporal feature fusion for facial expression recognition in static images

Sun, Ning; Li, Qi; Huan, Ruizhi; Liu, Jixin; Han, Guang

doi:10.1016/j.patrec.2017.10.022

Cited by 90 publications

(56 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, for the situations with unavailability of a neutral face, static techniques for FER have an edge over dynamic techniques. To circumvent this, "average human face" was suggested as an effective alternative representation of neutral face in a dynamic model [24]. Hitherto, there is no consensus about one technique being superior to the other comprehensively.…”

Section: Methods For Analysis Of the Described Features: Dynamic And mentioning

confidence: 99%

Dynamic Features Based on Flow-Correlation and HOG for Recognition of Discrete Facial Expressions

Anthwal¹,

Ganotra²

2020

RIA

View full text Add to dashboard Cite

Facial expressions are the most preeminent means of conveying one’s emotions and play a significant role in interpersonal communication. Researchers are in pursuit of endowing machines with the ability to interpret emotions from facial expressions as that will make human-computer interaction more efficient. With the objective of effective affect cognition from visual information, we present two dynamic descriptors that can recognise seven principal emotions. The variables of the appearance-based descriptor, FlowCorr, indicate intra-class similarity and inter-class difference by quantifying the degree of correlation of optical flow associated with the image pair and each pre-designed template describing the motion pattern associated with different expressions. The second shape-based descriptor, dyn-HOG, finds the HOG values of the difference image derived by subtracting neutral face from emotional face, and is demonstrated to be more discriminative than previously used static HOG descriptors for classifying facial expressions. Recognition accuracies with multi-class support vector machine obtained on the CK+ and KDEF-dyn datasets are competent with the results of state-of-the-art techniques and empirical analysis of human cognition of emotions.

show abstract

Section: Methods For Analysis Of the Described Features: Dynamic And mentioning

confidence: 99%

Dynamic Features Based on Flow-Correlation and HOG for Recognition of Discrete Facial Expressions

Anthwal¹,

Ganotra²

2020

RIA

View full text Add to dashboard Cite

show abstract

“…They found that emotion classification was very susceptible to head orientation and illumination variation. Sun et al [31] proposed a multi-channel deep neural network that learned and fused the spatial-temporal features. Lopes et al [32] proposed several pre-processing steps before feeding the faces to a convolutional neural network for facial expression recognition.…”

Section: Visual Signal Based Emotion Classificationmentioning

confidence: 99%

Facial Emotion Recognition Using Hybrid Features

Alreshidi

Ullah

2020

Informatics

View full text Add to dashboard Cite

Facial emotion recognition is a crucial task for human-computer interaction, autonomous vehicles, and a multitude of multimedia applications. In this paper, we propose a modular framework for human facial emotions’ recognition. The framework consists of two machine learning algorithms (for detection and classification) that could be trained offline for real-time applications. Initially, we detect faces in the images by exploring the AdaBoost cascade classifiers. We then extract neighborhood difference features (NDF), which represent the features of a face based on localized appearance information. The NDF models different patterns based on the relationships between neighboring regions themselves instead of considering only intensity information. The study is focused on the seven most important facial expressions that are extensively used in day-to-day life. However, due to the modular design of the framework, it can be extended to classify N number of facial expressions. For facial expression classification, we train a random forest classifier with a latent emotional state that takes care of the mis-/false detection. Additionally, the proposed method is independent of gender and facial skin color for emotion recognition. Moreover, due to the intrinsic design of NDF, the proposed method is illumination and orientation invariant. We evaluate our method on different benchmark datasets and compare it with five reference methods. In terms of accuracy, the proposed method gives 13% and 24% better results than the reference methods on the static facial expressions in the wild (SFEW) and real-world affective faces (RAF) datasets, respectively.

show abstract

“…These metrics driven by multiple features are subsequently learned with these extracted multiple features in a unified fashion to use complementary and discriminative data for emotion classification. Sun et al [20] introduced a multi-channel deep neural network that learns and puts together the spatialtemporal descriptors for facial expressions identification in static frames. The important concept of the algorithm is to discover and collect optical flow from the difference among the peak expression face frame and the neutral face frame as the temporal data of a specific facial expression, and consider the grey-level frame of peak expression face as the spatial data.…”

Section: Literature Reviewmentioning

confidence: 99%

Facial Emotion Recognition using Neighborhood Features

Aljaloud¹,

Ullah²,

Alanazi³

2020

IJACSA

View full text Add to dashboard Cite

We present a new method for human facial emotions recognition. For this purpose, initially, we detect faces in the images by using the famous cascade classifiers. Subsequently, we then extract a localized regional descriptor (LRD) which represents the features of a face based on regional appearance encoding. The LRD formulates and models various spatial regional patterns based on the relationships between local areas themselves instead of considering only raw and unprocessed intensity features of an image. To classify facial emotions into various classes of facial emotions, we train a multiclass support vector machine (M-SVM) classifier which recognizes these emotions during the testing stage. Our proposed method takes into account robust features and is independent of gender and facial skin color for emotion recognition. Moreover, our method is illumination and orientation invariant. We assessed our method on two benchmark datasets and compared it with four reference methods. Our proposed method outperformed them considering both the datasets.

show abstract

Deep spatial-temporal feature fusion for facial expression recognition in static images

Cited by 90 publications

References 12 publications

Dynamic Features Based on Flow-Correlation and HOG for Recognition of Discrete Facial Expressions

Dynamic Features Based on Flow-Correlation and HOG for Recognition of Discrete Facial Expressions

Facial Emotion Recognition Using Hybrid Features

Facial Emotion Recognition using Neighborhood Features

Contact Info

Product

Resources

About