Optimal Geometrical Set for Automated Marker Placement to Virtualized Real-Time Facial Emotions

Abstract

In recent years, real-time face recognition has been a major topic of interest in developing intelligent human-machine interaction systems. Over the past several decades, researchers have proposed different algorithms for facial expression recognition, but there has been little focus on detection in real-time scenarios. The present work proposes a new algorithmic method of automated marker placement used to classify six facial expressions: happiness, sadness, anger, fear, disgust, and surprise. Emotional facial expressions were captured using a webcam, while the proposed algorithm placed a set of eight virtual markers on each subject’s face. Facial feature extraction methods, including marker distance (distance between each marker to the center of the face) and change in marker distance (change in distance between the original and new marker positions), were used to extract three statistical features (mean, variance, and root mean square) from the real-time video sequence. The initial position of each marker was subjected to the optical flow algorithm for marker tracking with each emotional facial expression. Finally, the extracted statistical features were mapped into corresponding emotional facial expressions using two simple non-linear classifiers, K-nearest neighbor and probabilistic neural network. The results indicate that the proposed automated marker placement algorithm effectively placed eight virtual markers on each subject’s face and gave a maximum mean emotion classification rate of 96.94% using the probabilistic neural network.

Facial geometric feature extraction based emotional expression classification using machine learning algorithms

“…Some of the common issues in the previous works are: (a) most of the previous works used the international standard databases of static images with a limited number of samples (b) utilized fixed or reflective markers to track the facial expressions (c) most of the experiments are performed in a constrained environment with a limited number of emotional expressions, (d) utilized more extensive set of FAUs to detect facial expressions, and (e) variety of algorithms are proposed for recognizing facial expressions in an offline environment. To address the issues In [ 49 ] and [ 64 ], we have utilized a smaller number of samples for facial expression recognition and analyzed a simple distance measure to distinguish facial expressions. However, in the present work, we analyzed the facial emotional expressions of 85 subjects from five different nationalities (India, Kuwait, Malaysia, China, and Syria).…”

“…The face detection using this proposed method is faster than other methods in the literature (computation time: 0.067 sec) [ 47 ]. Finally, the algorithm proposed in our earlier work [ 47 ] formulates an ellipse around the face, positioned “+” markers on both eyes of the subject’s to ease the process of computer-generated marker placement [ 48 , 49 ]. A sample subject after face and eye detection is shown in Fig 3(a) and 3(b) , respectively.…”

“…We proposed a mathematical model for placing eight virtual markers on a defined location in the subject’s face [ 49 ]. Descriptions of the computer-generated marker placement algorithm for emotional facial expressions can be found in [ 49 ]. After the marker placement, each marker’s edges are used to formulate five triangles for facial emotional expression detection.…”

Emotion Recognition from Spatio-Temporal Representation of EEG Signals via 3D-CNN with Ensemble Learning Techniques

“…A non-physiological signal is composed of external signals such as gestures, facial expressions, verbal tones, etc. [ 3 ]. Most modern HCI systems still lack emotional intelligence and the ability to utilize these signals.…”