Facial Expression Recognition of Nonlinear Facial Variations Using Deep Locality De-Expression Residue Learning in the Wild

Abstract

Automatic facial expression recognition is an emerging field. Moreover, the interest has been increased with the transition from laboratory-controlled conditions to in the wild scenarios. Most of the research has been done over nonoccluded faces under the constrained environment, while automatic facial expression is less understood/implemented for partial occlusion in the real world conditions. Apart from that, our research aims to tackle the issues of overfitting (caused by the shortage of adequate training data) and to alleviate the expression-unrelated/intraclass/nonlinear facial variations, such as head pose estimation, eye gaze estimation, intensity and microexpressions. In our research, we control the magnitude of each Action Unit (AU) and combine several of the Action Unit combinations to leverage learning from the generative and discriminative representations for automatic FER. We have also addressed the problem of diversification of expressions from lab controlled to real-world scenarios from our cross-database study and proposed a model for enhancement of the discriminative power of deep features while increasing the interclass scatters, by preserving the locality closeness. Furthermore, facial expression consists of an expressive component as well as neutral component, so we proposed a generative model which is capable of generating neutral expression from an input image using cGAN. The expressive component is filtered and passed to the intermediate layers and the process is called De-expression Residue Learning. The residue in the intermediate/middle layers is very important for learning through expressive components. Finally, we validate the effectiveness of our method (DLP-DeRL) through qualitative and quantitative experimental results using four databases. Our method is more accurate and robust, and outperforms all the existing methods (hand crafted features and deep learning) while dealing the images in the wild.

Detection of multi‐class lung diseases based on customized neural network

“…46 presented GCNN for segmentation using diffusion‐weighted imaging data (DWI), while Sani et al 47 used it for the classification of breast cancer. In Ullah et al, 48 the DeepLungNet DL model, which comprises 20 learnable layers, that is, 18 convolution (ConV) layers including 2 group convolution layers and 2 FC layers, was proposed for COVID‐19 detection.…”

Enhancing explainability in brain tumor detection: A novel DeepEBTDNet model with LIME on MRI images

“…Unlike existing approaches, the proposed approach does not include segmentation, selection, or feature extraction in the pre‐processing phase 16,17 . High detection performance may be attained by the filter‐based feature extraction method used in the proposed model 18,19 . The proposed model extracts high‐level information from the MR images by combining a convolutional layer with the LReLU activation function.…”

Empirical Investigation of Multimodal Sensors in Novel Deep Facial Expression Recognition In‐the‐Wild

“…Facial expression recognition is used in various domains like Intelligent Tutoring System (ITS), psychology, human-machine interaction, behavioral science, intelligent transportation, and interactive games [4]. It can be helpful in monitoring the abnormal expressions in the crowd at public places to avoid any crime.…”