Facial Emotion Recognition Combining Auxiliary Classifiers and Multiscale CBAM Attention Mechanisms

Abstract

Facial emotions are the most intuitive way to react to changes in inner emotions. We propose a facial emotion recognition method that combines auxiliary classifiers(Acs) and multi-scale CBAM(MCBAM) by improving the Xception network model. And we design a lightweight network model AMDCNN. We introduce Acs in the middle layers of the model. The features extracted from the middle layer portion of the model are utilized to aid in emotion recognition. Finally, the recognition results of the Acs and the main classifier are adaptively weighted and fused to obtain better emotion recognition results. This enables better utilization of the feature information extracted from the intermediate layers and further reduces the feature loss caused by the downsampling process of the convolutional layer. The CBAM does not increase the number of parameters and computation too much, and it enables the model to better focus on the important areas of the face. We apply it to the proposed lightweight model and improve it further. The width is increased by introducing a multi-branch convolutional structure and utilizing convolutional layers with different kernel sizes. This allows for more adequate spatial and channel features during feature extraction, allowing the model to more accurately focus on important facial regions. Our proposed model was experimentally validated on datasets of FER2013, FERPlus, RAF-DB and CK+, with accuracies of 69.82%, 85.40%, 86.77% and 99.49%, respectively. The number of parameters of the proposed model is only 1.6M. Our model is a good competitive advantage compared with other lightweight models.