Facial Expression Recognition Based on Spatial and Channel Attention Mechanisms

Yao, Lisha; He, Simin; Su, Kang; Shao, Qingtong

doi:10.1007/s11277-022-09616-y

Cited by 11 publications

(3 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Li et al [17] proposed a novel depth emotion conditional adaptation network (ECAN), which achieves effective alignment of global marginal distributions and end-to-end matching of fine-grained class conditional distributions by fully utilizing low-level label information in the target data, aiming to address the issue of neglected imbalances in emotion category distributions. Yao et al [18] embedded the HPMI attention module into the VGG-16 network to elevate the importance of critical features, mitigating overfitting issues and enabling accurate recognition of subtle facial expression variations in real-world scenarios. While these deep learning methods enhance the capability of extracting facial expression features by employing larger and deeper networks, they also increase network parameters and computational complexity, thereby reducing network recognition efficiency.…”

Section: Related Workmentioning

confidence: 99%

Enhanced Hybrid Vision Transformer with Multi-Scale Feature Integration and Patch Dropping for Facial Expression Recognition

Li,

Huang,

Wang

et al. 2024

Sensors

View full text Add to dashboard Cite

Convolutional neural networks (CNNs) have made significant progress in the field of facial expression recognition (FER). However, due to challenges such as occlusion, lighting variations, and changes in head pose, facial expression recognition in real-world environments remains highly challenging. At the same time, methods solely based on CNN heavily rely on local spatial features, lack global information, and struggle to balance the relationship between computational complexity and recognition accuracy. Consequently, the CNN-based models still fall short in their ability to address FER adequately. To address these issues, we propose a lightweight facial expression recognition method based on a hybrid vision transformer. This method captures multi-scale facial features through an improved attention module, achieving richer feature integration, enhancing the network’s perception of key facial expression regions, and improving feature extraction capabilities. Additionally, to further enhance the model’s performance, we have designed the patch dropping (PD) module. This module aims to emulate the attention allocation mechanism of the human visual system for local features, guiding the network to focus on the most discriminative features, reducing the influence of irrelevant features, and intuitively lowering computational costs. Extensive experiments demonstrate that our approach significantly outperforms other methods, achieving an accuracy of 86.51% on RAF-DB and nearly 70% on FER2013, with a model size of only 3.64 MB. These results demonstrate that our method provides a new perspective for the field of facial expression recognition.

show abstract

Section: Related Workmentioning

confidence: 99%

Enhanced Hybrid Vision Transformer with Multi-Scale Feature Integration and Patch Dropping for Facial Expression Recognition

Li,

Huang,

Wang

et al. 2024

Sensors

View full text Add to dashboard Cite

show abstract

“…DenseNet [21] connects each layer with other layers in a feed-forwardly.The DenseNet network consists of dense blocks calculated as shown in Equation (1).…”

Section: B Improved Deep Neural Network Model With Mixed Attention Me...mentioning

confidence: 99%

Emotion Recognition in Complex Classroom Scenes Based on Improved Convolutional Block Attention Module Algorithm

Li,

Yao

2023

IEEE Access

View full text Add to dashboard Cite

This study provides a deep learning-based intelligent recognition technology for student facial expressions in the classroom. This technology realizes the recognition of students' facial expressions and provides a practical approach for classroom assessment and teacher improvement of teaching methods toward achieving smart education. An improved hybrid attention mechanism is designed for the student classroom. The facial expression recognition model addresses issues of instability in the recognition process of student facial expressions in the classroom, the high redundancy of parameters in traditional convolutional neural networks, and the long training time and slow convergence prone to overfitting. In the image modality data, this study proposes a hybrid attention mechanism in the deep neural network model before feature fusion to extract network features with stronger representational capability, enhance the prediction performance of deep neural networks, and improve the interpretability of the model. The improved hybrid attention mechanism is introduced into the deep neural network by modifying the convolutional block attention module with shortcut connections, deepening the network depth of the attention module and enabling it to learn the weight information among feature channels and spatial regions effectively. The proposed student facial expression recognition model achieves an accuracy of 88.71% on the publicly available RAF-DB dataset and an accuracy of 86.14% on the self-collected real classroom teaching video dataset. The proposed technology can be applied in the education field to evaluate student engagement automatically, provide personalized teaching guidance and learning analytics for teachers, and promote the development of intelligent education.

show abstract

“…CBAM is a lightweight module that combines channel attention and spatial attention to dramatically improve model performance while requiring a small amount of computation and a small number of parameters. The channel attention mechanism [16][17][18] focuses on which channel features are meaningful using global average pooling and global maximum pooling to obtain two feature maps and then feeds them sequentially into a weight-sharing multilayer perceptron with a 1 × 1 convolution to better fuse channel information. The spatial attention mechanism [19,20] focuses on spatial features, mainly on the part of the input image that is richer in effective information.…”

Section: Related Workmentioning

confidence: 99%

Multiscale Feature Fusion Attention Lightweight Facial Expression Recognition

Zhang

2022

International Journal of Aerospace Engineering

View full text Add to dashboard Cite

Facial expression recognition based on residual networks is important for technologies related to space human-robot interaction and collaboration but suffers from low accuracy and slow computation in complex network structures. To solve these problems, this paper proposes a multiscale feature fusion attention lightweight wide residual network. The network first uses an improved random erasing method to preprocess facial expression images, which improves the generalizability of the model. The use of a modified depthwise separable convolution in the feature extraction network reduces the computational effort associated with the network parameters and enhances the characterization of the extracted features through a channel shuffle operation. Then, an improved bottleneck block is used to reduce the dimensionality of the upper layer network feature map to further reduce the number of network parameters while enhancing the network feature extraction capability. Finally, an optimized multiscale feature lightweight attention mechanism module is embedded to further improve the feature extractability of the network for human facial expressions. The experimental results show that the accuracy of the model is 73.21%, 98.72%, and 95.21% on FER2013, CK+ and JAFFE, respectively, with a covariance of 10.14 M. Compared with other networks, the model proposed in this paper has faster computing speed and better accuracy at the same time.

show abstract

Facial Expression Recognition Based on Spatial and Channel Attention Mechanisms

Cited by 11 publications

References 25 publications

Enhanced Hybrid Vision Transformer with Multi-Scale Feature Integration and Patch Dropping for Facial Expression Recognition

Enhanced Hybrid Vision Transformer with Multi-Scale Feature Integration and Patch Dropping for Facial Expression Recognition

Emotion Recognition in Complex Classroom Scenes Based on Improved Convolutional Block Attention Module Algorithm

Multiscale Feature Fusion Attention Lightweight Facial Expression Recognition

Contact Info

Product

Resources

About