Facial Expression Recognition Using Deep Siamese Neural Networks with a Supervised Loss function

Hayale, Wassan; Negi, Pooran; Mahoor, Mohammad H.

doi:10.1109/fg.2019.8756571

Cited by 23 publications

(23 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several reports have shown that these machine learning classifiers are mainly used nowadays to produce emotion categorization devices (Alenazy & Alqahtani, 2021;Choi & Song, 2020;Hayale et al, 2019;Zhang et al, 2019). However, the findings of this research provide insights that these classifiers do not perform well in categorizing emotion when the face is covered.…”

Section: Discussionmentioning

confidence: 99%

A Comparison of Humans and Machine Learning Classifiers Detecting Emotion from Faces of People with Different Coverings

Shehu¹,

Wn²,

Eisenbarth³

2021

Preprint

View full text Add to dashboard Cite

Partial face coverings such as sunglasses and facemasks have now become the ‘new norm’, especially since the increase of infectious diseases. Unintentionally, they obscure facial expressions. Therefore, humans and artificial systems have been found to be less accurate in emotion categorization. However, it is unknown how similar the performance of humans compared with artificial systems is affected based on the exact same stimuli, varying systematically in types of coverings. Such a systematic direct comparison would allow conclusions about the relevant facial features in a naturalistic context. Therefore, we investigated the impact of facemasks and sunglasses on the ability to categorize emotional facial expressions in humans and artificial systems. Artificial systems, represented by the VGG19 deep learning algorithm, and humans assessed images of people with varying emotional facial expressions and with four different types of coverings, i.e. unmasked (original images), mask (mask covering lower-face), partial mask (with transparent mouth window), and sunglasses. Artificial systems performed significantly better than humans when no covering is present (> 15% difference). However, the achieved accuracy of both humans and artificial systems differed significantly depending on the type of coverings and, importantly, emotion, e.g. the use of sunglasses reduced accuracy for recognition of fear in humans. It was also noted that while humans mainly classify unknown expressions as neutral across all coverings, the misclassification varied in the artificial systems. These findings show humans and artificial systems classify and misclassify various emotion expressions differently depending on both the type of face covering and type of emotion.

show abstract

Section: Discussionmentioning

confidence: 99%

A Comparison of Humans and Machine Learning Classifiers Detecting Emotion from Faces of People with Different Coverings

Shehu¹,

Wn²,

Eisenbarth³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…These neural networks are identical and their outputs are connected using a differentiator. The output of this differentiator, L(o 1 , o 2 ), is then used as a loss function during the training of the SNN [28].…”

Section: Siamese Neural Networkmentioning

confidence: 99%

Neural Network-Based Model for Classification of Faults During Operation of a Robotic Manipulator

et al. 2021

View full text Add to dashboard Cite

The importance of error detection is high, especially in modern manufacturing processes where assembly lines operate without direct supervision. Stopping the faulty operation in time can prevent damage to the assembly line. Public dataset is used, containing 15 classes, 2 types of faultless operation and 13 types of faults, with 463 force and torsion datapoints. Four different methods are used: Multilayer Perceptron (MLP) selected due to high classification performance, Support Vector Machines (SVM) commonly used for a low number of datapoints, Convolutional Neural Network (CNN) known for high performance in classification with matrix inputs and Siamese Neural Network (SNN) novel method with high performance in small datasets. Two classification tasks are performed-error detection and classification. Grid search is used for hyperparameter variation and F1 score as a metric, with a 10 fold cross-validation. Authors propose a hybrid system consisting of SNN for detection and CNN for fault classification.

show abstract

“…The quality and appearance of input face images to the CNNs can vary in different ways. Specifically, the face resolution varies considerably across the literature, typically ranging from small resolutions such as 48×48 [5,6,7] to large ones such as 256×256 [8,9,10]). The face resolution affects the computational complexity and potentially the model performance.…”

Section: Introductionmentioning

confidence: 99%

“…In addition, some researchers [5,6,11] converted the color (RGB) images to grayscale, while others [12,13] employed the RGB images. Also, many works localized and aligned the face images before feature extraction [12,6,9], while some studies used the images without alignment [13,14]. Lastly, some scholars used the full face images as input to the CNNs [5,6,9], while others cropped AU regions [11,15].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Choose Settings Carefully: Comparing Action Unit detection at Different Settings Using a Large-Scale Dataset

Bishay¹,

Ghoneim²,

Ashraf³

et al. 2021

Preprint

View full text Add to dashboard Cite

In this paper, we investigate the impact of some of the commonly used settings for (a) preprocessing face images, and (b) classification and training, on Action Unit (AU) detection performance and complexity. We use in our investigation a large-scale dataset, consisting of ∼55K videos collected in the wild for participants watching commercial ads. The preprocessing settings include scaling the face to a fixed resolution, changing the color information (RGB to gray-scale), aligning the face, and cropping AU regions, while the classification and training settings include the kind of classifier (multi-label vs. binary) and the amount of data used for training models. To the best of our knowledge, no work had investigated the effect of those settings on AU detection. In our analysis we use CNNs as our baseline classification model.

show abstract

Facial Expression Recognition Using Deep Siamese Neural Networks with a Supervised Loss function

Cited by 23 publications

References 20 publications

A Comparison of Humans and Machine Learning Classifiers Detecting Emotion from Faces of People with Different Coverings

A Comparison of Humans and Machine Learning Classifiers Detecting Emotion from Faces of People with Different Coverings

Neural Network-Based Model for Classification of Faults During Operation of a Robotic Manipulator

Choose Settings Carefully: Comparing Action Unit detection at Different Settings Using a Large-Scale Dataset

Contact Info

Product

Resources

About