Facial Feature Extraction Using a Symmetric Inline Matrix-LBP Variant for Emotion Recognition

Abstract

With a large number of Local Binary Patterns (LBP) variants being currently used today, the significant and importance of visual descriptors in computer vision applications are prominent. This paper presents a novel visual descriptor, i.e., SIM-LBP. It employs a new matrix technique called the Symmetric Inline Matrix generator method, which acts as a new variant of LBP. The key feature that separates our variant from existing counterparts is that our variant is very efficient in extracting facial expression features like eyes, eye brows, nose and mouth in a wide range of lighting conditions. For testing our model, we applied SIM-LBP on the JAFFE dataset to convert all the images to its corresponding SIM-LBP transformed variant. These transformed images are then used to train a Convolution Neural Network (CNN) based deep learning model for facial expressions recognition (FER). Several performance evaluation metrics, i.e., recognition accuracy rate, precision, recall, and F1-score, were used to test mode efficiency in comparison with those using the traditional LBP descriptor and other LBP variants. Our model outperformed in all four matrices with the proposed SIM-LBP transformation on the input images against those of baseline methods. In comparison analysis with the other state-of-the-art methods, it shows the usefulness of the proposed SIM-LBP model. Our proposed SIM-LBP variant transformation can also be applied on facial images to identify a person’s mental states and predict mood variations.

Assessment of a Person’s Emotional State Based on His or Her Posture Parameters

“…This limits the use of these methods, for example, in systems controlling the interaction of a person with technical objects. The second group of methods is based on behavioral reactions assessed by facial features, such as mouth activity, head movements, blink frequency, spatial distribution of gaze, pupil dilation, and eye movements [ 20 , 21 , 22 , 23 ]; voice [ 24 , 25 , 26 ]; and movements, gait, and body postures [ 27 , 28 , 29 ]. Currently, research on emotion recognition has mainly focused on facial expression and physiological cues, while emotion recognition based on the modality of posture has been investigated little.…”

Feature Extraction Techniques for Facial Expression Recognition (FER)

“…Binary Patterns (LBP), a popular local descriptor, has gained attention in the scientific community for its role in feature extraction. As one of the earliest descriptors, LBP has evolved into various variants, shaping the field of feature extraction and pattern recognition [4]. Here are additional specifics regarding the significance of feature extraction in facial expression recognition:…”