Fusing multi-stream deep neural networks for facial expression recognition

Salmam, Fatima Zahra; Madani, Abdellah; Kissi, Mohamed

doi:10.1007/s11760-018-1388-4

Cited by 16 publications

(8 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These hand-designed features rely on a large amount of expert experience, have good interpretability, and can be used in various tasks in the field of computer vision. 14 For example, Gabor wavelet can capture the edge information of images at different scales and directions, and has certain robustness to image rotation, deformation, and illumination changes; Local Binary Pattern (LBP) has rotation invariance and gray invariance to images; Histogram of Oriented Gradient has good invariance to the geometric and optical deformation of the image . 15 Reference 16 uses partitioning technology for feature extraction, combined with Hidden Markov Model (HMM) extension to classify facial expression features.…”

Section: Related Researchmentioning

confidence: 99%

Multi‐angle face expression recognition based on generative adversarial networks

2021

Computational Intelligence

View full text Add to dashboard Cite

Because of the different features of the same facial expressions from different angles, most of the methods are only suitable for face images, and the accuracy of facial expression recognition is low. Therefore, a multi-angle facial expression recognition method based on generative adversarial networks (GAN) is proposed.Firstly, the depth regression network is used to detect the key points of the face image to achieve face alignment, so as to reduce the difficulty of feature extraction. Then, the image is input to GAN. The generator is composed of encoder and decoder, and a skip connection is designed.In the encoding phase, the generator can unlock the correlation between the facial expression image and the angle, and in the decoding stage, it can generate different angle facial expression images by adding other angle information. Finally, the multi-angle facial expression images are sent to the convolution neural network for classification and learning, in which the loss weight is adjusted dynamically to improve the recognition accuracy by introducing resistance loss, recognition loss, content loss, and center loss. The experimental results on Multi-pose illumination expression (PIE) and celebrities in frontal profile (CFP) datasets show that the performance of the proposed method is the best when the learning rate is 0.0002, and the recognition accuracy 20

show abstract

Section: Related Researchmentioning

confidence: 99%

Multi‐angle face expression recognition based on generative adversarial networks

2021

Computational Intelligence

View full text Add to dashboard Cite

show abstract

“…They exploited both spatial and temporal dimensions to achieve competitive recognition accuracies on benchmark datasets. Siddiqi et al [25] presented an offline FER architecture that used stepwise linear discriminant analysis complemented with hidden conditional random fields model. The former extracted relevant features from input expression images with the help of partial F-test values to curtail intraclass differences and inflate inter-class variation.…”

Section: Related Workmentioning

confidence: 99%

“…The latter, adept to approximate complex distributions with Gaussian density functions, was used for classification of the extracted features. Salmam et al [25] presented a hybrid model that coupled a CNN representing appearance-based features such as wrinkles and skin folds and a deep neural network based on geometric features characterizing salient facial parts such as eyes, nose, and mouth. They demonstrated the increase in efficiency of FER by integrating both types of features.…”

Section: Related Workmentioning

confidence: 99%

Modelling and Analysis of Facial Expressions Using Optical Flow Derived Divergence and Curl Templates

Anthwal

2021

ELCVIA

View full text Add to dashboard Cite

Facial expressions are integral part of non-verbal paralinguistic communication as they provide cues significant in perceiving one’s emotional state. Assessment of emotions through expressions is an active research domain in computer vision due to its potential applications in multi-faceted domains. In this work, an approach is presented where facial expressions are modelled and analyzed with dense optical flow derived divergence and curl templates that embody the ideal motion pattern of facial features pertaining to unfolding of an expression on the face. Two types of classification schemes based on multi-class support vector machine and k-nearest neighbour are employed for evaluation. Promising results obtained from comparative analysis of the proposed approach with state-of-the-art techniques on the Extended Cohn Kanade database and with human cognition and pre-trained Microsoft face application programming interface on the Karolinska Directed Emotional Faces database validate the efficiency of the approach.

show abstract

“…Happy and Routray [18] integrated both the features in a model that used local binary patterns for analysing texture information extracted from salient facial patches. Salmam et al [19] combined a CNN architecture representing appearance features and a Deep Neural Network (DNN) framework based on geometric features to demonstrate that integrating features increased the efficiency of FER method.…”

Section: Facial Geometry-or Appearance-based Featuresmentioning

confidence: 99%

“…[46] 92. 5 Meena et al (2020) [47] 92.9 Wei et al (2020) [39] 94.4 Makhmudkhujaev et al (2019) [48] 94.5 Cheng and Zhou (2020) [49] 96.0 Chen and Hu (2020) [50] 96.3 Gan et al (2020) [38] 96.3 De la torre et al (2015) [51] 96.4 Qin et al(2020) [40] 96.8 Dyn-HOG (with multi-class SVM) 96.8 Salmam et al (2019) [19] 96.9 Li et al (2020) [52] 97. 4 Zhao et al (2018) [32] with optical flow 97.5 FlowCorr (with mult-iclass SVM) 98.0 Sadeghi and Raie (2019) [53] 98.2 Table 10 encapsulates percentage accuracy comparison for each emotion of the presented descriptors with a perceptual study of human interpretation of basic emotions conducted by Calvo et al [9] on KDEF-dyn dataset.…”

Section: Recognition Performance Of Dyn-hog Descriptormentioning

confidence: 99%

Dynamic Features Based on Flow-Correlation and HOG for Recognition of Discrete Facial Expressions

Anthwal¹,

Ganotra²

2020

RIA

View full text Add to dashboard Cite

Facial expressions are the most preeminent means of conveying one’s emotions and play a significant role in interpersonal communication. Researchers are in pursuit of endowing machines with the ability to interpret emotions from facial expressions as that will make human-computer interaction more efficient. With the objective of effective affect cognition from visual information, we present two dynamic descriptors that can recognise seven principal emotions. The variables of the appearance-based descriptor, FlowCorr, indicate intra-class similarity and inter-class difference by quantifying the degree of correlation of optical flow associated with the image pair and each pre-designed template describing the motion pattern associated with different expressions. The second shape-based descriptor, dyn-HOG, finds the HOG values of the difference image derived by subtracting neutral face from emotional face, and is demonstrated to be more discriminative than previously used static HOG descriptors for classifying facial expressions. Recognition accuracies with multi-class support vector machine obtained on the CK+ and KDEF-dyn datasets are competent with the results of state-of-the-art techniques and empirical analysis of human cognition of emotions.

show abstract

Fusing multi-stream deep neural networks for facial expression recognition

Cited by 16 publications

References 26 publications

Multi‐angle face expression recognition based on generative adversarial networks

Multi‐angle face expression recognition based on generative adversarial networks

Modelling and Analysis of Facial Expressions Using Optical Flow Derived Divergence and Curl Templates

Dynamic Features Based on Flow-Correlation and HOG for Recognition of Discrete Facial Expressions

Contact Info

Product

Resources

About