A Novel Model for Emotion Detection from Facial Muscles Activity

Abstract

Considering human's emotion in different applications and systems has received substantial attention over the last three decades. The traditional approach for emotion detection is to first extract different features and then apply a classifier, like SVM, to find the true class. However, recently proposed Deep Learning based models outperform traditional machine learning approaches without requirement of a separate feature extraction phase. This paper proposes a novel deep learning based facial emotion detection model, which uses facial muscles activities as raw input to recognize the type of the expressed emotion in the real time. To this end, we first use OpenFace to extract the activation values of the facial muscles, which are then presented to a Stacked Auto Encoder (SAE) as feature set. Afterward, the SAE returns the best combination of muscles in describing a particular emotion, these extracted features at the end are applied to a Softmax layer in order to fulfill multi classification task. The proposed model has been applied to the CK+, MMI and RADVESS datasets and achieved respectively average accuracies of 95.63%, 95.58%, and 84.91% for emotion type detection in six classes, which outperforms state-of-the-art algorithms.

Privacy-Preserving Video Classification with Convolutional Neural Networks

“…After this replacement, the accuracy of the video classifier is 56.8%. This is in line with state-of-the-art results in the literature on emotion recognition from RAVDESS videos, namely 57.5% with Synchronous Graph Neural Networks (8 emotions) [50]; 61% with ConvNet-LSTM (8 emotions) [1]; 59% with an RNN (7 emotions) [9], and 82.4% with stacked autoencoders (6 emotions) [5].…”

“…V, has been studied by other authors in-the-clear, i.e. without regards for privacy protection, using a variety of deep learning architectures, with reported accuracies in the 57%-82% range, depending on the number of emotion classes included in the study (6 to 8) [5], [50], [9], [1]. The ConvNet model that we trained for our experimental results in Sec.…”

Facial expression recognition based on adaptation of the classifier to videos of the user

“…Поэтому в предыдущей работе авторов настоящего исследования [13] и в ряде других статей [14,15] набор данных RAVDESS разделяется по актёрам, когда все видео части актёров попадают в обучающее множество, а видео оставшихся людей -в тестовую выборку. При таком подходе к разделению на тестовое и тренировочное множества, каждое из множеств содержит только уникальных актёров.…”

“…Например, в статье [14] набор данных был разделён по актёрам в следующем соотношении: 75 % актёров в тренировочном наборе и 25 % в тестовом. Авторы использовали модель активности лицевых мышц для распознавания типа эмоции в режиме реального времени.…”