A Robust Facial Expression Recognition Algorithm Based on Multi-Rate Feature Fusion Scheme

Abstract

In recent years, the importance of catching humans’ emotions grows larger as the artificial intelligence (AI) field is being developed. Facial expression recognition (FER) is a part of understanding the emotion of humans through facial expressions. We proposed a robust multi-depth network that can efficiently classify the facial expression through feeding various and reinforced features. We designed the inputs for the multi-depth network as minimum overlapped frames so as to provide more spatio-temporal information to the designed multi-depth network. To utilize a structure of a multi-depth network, a multirate-based 3D convolutional neural network (CNN) based on a multirate signal processing scheme was suggested. In addition, we made the input images to be normalized adaptively based on the intensity of the given image and reinforced the output features from all depth networks by the self-attention module. Then, we concatenated the reinforced features and classified the expression by a joint fusion classifier. Through the proposed algorithm, for the CK+ database, the result of the proposed scheme showed a comparable accuracy of 96.23%. For the MMI and the GEMEP-FERA databases, it outperformed other state-of-the-art models with accuracies of 96.69% and 99.79%. For the AFEW database, which is known as one in a very wild environment, the proposed algorithm achieved an accuracy of 31.02%.

A Hybrid Model for Driver Emotion Detection Using Feature Fusion Approach

“…We have compared our proposed hybrid model with recent state-of-the-art FER methods, including Inception-Resnet and LSTM [ 35 ], DCMA-CNN [ 27 ], WRF [ 22 ], LMRF [ 24 ], VGG11+SVM [ 40 ], DNN+RELM [ 43 ], LBP+ORB+SVM [ 25 ], and MDNETWORK [ 33 ] on the CK+ dataset. These works adopted machine learning algorithms and deep neural networks in combined manner or individually used.…”

“…In the remaining works, Inception-Resnet and LSTM [ 35 ], DNN+RELM [ 43 ], VGG11+SVM [ 40 ] have used both deep neural networks and classifiers along with handcrafted features extracted by machine learning. DCMA-CNN [ 27 ] and MDNETWORK [ 33 ] were implemented using multi branch convolutional neural networks. Our proposed hybrid approach outperforms the state-of-the-art models with an achieved accuracy of 95.1% listed in Table 4 , except for MDNETWORK [ 33 ] which is 1.1% greater than our accuracy.…”

“…DCMA-CNN [ 27 ] and MDNETWORK [ 33 ] were implemented using multi branch convolutional neural networks. Our proposed hybrid approach outperforms the state-of-the-art models with an achieved accuracy of 95.1% listed in Table 4 , except for MDNETWORK [ 33 ] which is 1.1% greater than our accuracy.…”

“…S.J. Park et al, 2021, [ 33 ] proposed a robust facial expression recognition algorithm based on a multi-rate feature fusion scheme on multi-depth CNN, achieving an accuracy of 96.23% on the CK+ dataset. S.H.…”

Presentation Attacks in Palmprint Recognition Systems

“…In recent years, biometric recognition has been widely concerned and applied. Biometric recognition includes palmprint [1], face [2][3], gesture [4], and other modalities.…”