FER-BHARAT: a lightweight deep learning network for efficient unimodal facial emotion recognition in Indian context

Abstract

Humans' ability to manage their emotions has a big impact on their ability to plan and make decisions. In order to better understand people and improve human–machine interaction, researchers in affective computing and artificial intelligence are investigating the detection and recognition of emotions. However, different cultures have distinct ways of expressing emotions, and the existing emotion recognition datasets and models may not effectively capture the nuances of the Indian population. To address this gap, this study proposes custom-built lightweight Convolutional Neural Network (CNN) models that are optimized for accuracy and computational efficiency. These models are trained and evaluated on two Indian emotion datasets: The Indian Spontaneous Expression Dataset (ISED) and the Indian Semi Acted Facial Expression Database (iSAFE). The proposed CNN model with manual feature extraction provides remarkable accuracy improvement of 11.14% for ISED and 4.72% for iSAFE datasets as compared to baseline, while reducing the training time. The proposed model also surpasses the accuracy produced by pre-trained ResNet-50 model by 0.27% ISED and by 0.24% for the iSAFE dataset with significant improvement in training time of approximately 320 s for ISED and 60 s for iSAFE dataset. The suggested lightweight CNN model with manual feature extraction offers the advantage of being computationally efficient and more accurate compared to pre-trained model making it a more practical and efficient solution for emotion recognition among Indians.

Method of Multi-Label Visual Emotion Recognition Fusing Fore-Background Features

“…These methods [8][9][10]17] have achieved superior performance by developing effective facial feature extraction networks compared to traditional methods such as CPR [18] and LBF [19]. Siqueira et al [8] and Karani et al [10] optimized their CNN models for computational efficiency, resulting in models with both high computational efficiency and accuracy.…”

“…These methods [8][9][10]17] have achieved superior performance by developing effective facial feature extraction networks compared to traditional methods such as CPR [18] and LBF [19]. Siqueira et al [8] and Karani et al [10] optimized their CNN models for computational efficiency, resulting in models with both high computational efficiency and accuracy. Liao et al [9] enhanced emotion recognition by incorporating facial optical flow information, while Arabian et al [17] constructed a facial key point grid and utilized GCN for emotion classification.…”

“…The methods proposed in this paper were compared with ML-KNN [13], ML-RBF [14], Kosti et al [11], Lee et al [21], Zhang et al [22], ML-GCN [27], ESRs [8], Ilyes et al [23], Q2L [15], LGLM [28], RCL-Net [9], ML-Decoder [16], SAML [26], FLNet [29], BHARAT [10], and LFPLM [17]. The comparison results of mAP and JC are shown in Table 5 and Figure 9.…”

“…Current emotion recognition research [8][9][10] predominantly focuses on single-label recognition of a few common emotions, such as Happy, Sad, Surprise, Neutral, and Angry. However, human emotions are complex and multidimensional, and single-label classification cannot capture the richness of emotional expression.…”