Video Based Emotion Recognition Using CNN and BRNN

Cai, Youyi; Zheng, Wenming; Zhang, Tong; Li, Qiang; Cui, Zhen; Ye, Jiayin

doi:10.1007/978-981-10-3005-5_56

Cited by 16 publications

(4 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such manipulations include horizontal flipping, cropping, rotation, translations, changes to color, brightness, and saturation, as well as scaling. This way, researchers artificially increase the number of available examples or training epochs by a factor typically between 10 and 30 [88], [113], [114], and up to 300 [115]. Studies running experiments on this technique report accuracy improvements (e.g., from 79% to 89% in [87]; see also [74], [94]).…”

Section: Learning Spatial Features For Fermentioning

confidence: 99%

Deep Learning for Human Affect Recognition: Insights and New Developments

Rouast

Adam

Chiong

2021

IEEE Trans. Affective Comput.

153

View full text Add to dashboard Cite

Automatic human affect recognition is a key step towards more natural human-computer interaction. Recent trends include recognition in the wild using a fusion of audiovisual and physiological sensors, a challenging setting for conventional machine learning algorithms. Since 2010, novel deep learning algorithms have been applied increasingly in this field. In this paper, we review the literature on human affect recognition between 2010 and 2017, with a special focus on approaches using deep neural networks. By classifying a total of 950 studies according to their usage of shallow or deep architectures, we are able to show a trend towards deep learning. Reviewing a subset of 233 studies that employ deep neural networks, we comprehensively quantify their applications in this field. We find that deep learning is used for learning of (i) spatial feature representations, (ii) temporal feature representations, and (iii) joint feature representations for multimodal sensor data. Exemplary state-of-the-art architectures illustrate the progress. Our findings show the role deep architectures will play in human affect recognition, and can serve as a reference point for researchers working on related applications.

show abstract

Section: Learning Spatial Features For Fermentioning

confidence: 99%

Deep Learning for Human Affect Recognition: Insights and New Developments

Rouast

Adam

Chiong

2021

IEEE Trans. Affective Comput.

153

View full text Add to dashboard Cite

show abstract

“…Figure 3 depicts a grayscale image that has been converted. These images are shades ranging from dark to light and black to white [23]. A grayscale image is created by determining the light intensity in each pixel over a specified electromagnetic range (for example, infrared or visible light).…”

Section: Dataset Preparationmentioning

confidence: 99%

Enhanced Emotion Recognition in Videos: A Convolutional Neural Network Strategy for Human Facial Expression Detection and Classification

Ashraf¹,

Gunawan

Arifin³

et al. 2023

IJEEI

View full text Add to dashboard Cite

The human face is essential in conveying emotions, as facial expressions serve as effective, natural, and universal indicators of emotional states. Automated emotion recognition has garnered increasing interest due to its potential applications in various fields, such as human-computer interaction, machine learning, robotic control, and driver emotional state monitoring. With artificial intelligence and computational power advancements, visual emotion recognition has become a prominent research area. Despite extensive research employing machine learning algorithms like convolutional neural networks (CNN), challenges remain concerning input data processing, emotion classification scope, data size, optimal CNN configurations, and performance evaluation. To address these issues, we propose a comprehensive CNN-based model for real-time detection and classification of five primary emotions: anger, happiness, neutrality, sadness, and surprise. We employ the Amsterdam Dynamic Facial Expression Set -Bath Intensity Variations (ADFES-BIV) video dataset, extracting image frames from the video samples. Image processing techniques such as histogram equalization, color conversion, cropping, and resizing are applied to the frames before labeling. The Viola-Jones algorithm is then used for face detection on the processed grayscale images. We develop and train a CNN on the processed image data, implementing dropout, batch normalization, and L2 regularization to reduce overfitting. The ideal hyperparameters are determined through trial and error, and the model's performance is evaluated. The proposed model achieves a recognition accuracy of 99.38%, with the confusion matrix, recall, precision, F1 score, and processing time further quantifying its performance characteristics. The model's generalization performance is assessed using images from the Warsaw Set of Emotional Facial Expression Pictures (WSEFEP) and Extended Cohn-Kanade Database (CK+) datasets. The results demonstrate the efficiency and usability of our proposed approach, contributing valuable insights into real-time visual emotion recognition.

show abstract

“…Many researchers have implemented different training architectures of CNN to improve their recognition accuracies. In [18,19], CNN was trained with many CNN architectures obtained from the pre trained model (ImageNet) as shown in Table 2. The results become more promising with the advent of more and more researches.…”

Section: Related Research Workmentioning

confidence: 99%

Development of video-based emotion recognition using deep learning with Google Colab

Gunawan

Ashraf

Riza³

et al. 2020

TELKOMNIKA

View full text Add to dashboard Cite

Emotion recognition using images, videos, or speech as input is considered as a hot topic in the field of research over some years. With the introduction of deep learning techniques, e.g., convolutional neural networks (CNN), applied in emotion recognition, has produced promising results. Human facial expressions are considered as critical components in understanding one's emotions. This paper sheds light on recognizing the emotions using deep learning techniques from the videos. The methodology of the recognition process, along with its description, is provided in this paper. Some of the video-based datasets used in many scholarly works are also examined. Results obtained from different emotion recognition models are presented along with their performance parameters. An experiment was carried out on the fer2013 dataset in Google Colab for depression detection, which came out to be 97% accurate on the training set and 57.4% accurate on the testing set.

show abstract

Video Based Emotion Recognition Using CNN and BRNN

Cited by 16 publications

References 22 publications

Deep Learning for Human Affect Recognition: Insights and New Developments

Deep Learning for Human Affect Recognition: Insights and New Developments

Enhanced Emotion Recognition in Videos: A Convolutional Neural Network Strategy for Human Facial Expression Detection and Classification

Development of video-based emotion recognition using deep learning with Google Colab

Contact Info

Product

Resources

About