Facial expression recognition using depth information and spatiotemporal features

Abstract

Facial expression recognition is considered to be one of the very important topics in image processing, pattern recognition, and computer vision due to its broad applications such as human-computer interaction, behavior analysis, and image understanding. In this work, a novel approach is proposed to recognize some facial expressions from time-sequential depth videos. First of all, efficient Local Directional Pattern (LDP) features are obtained from the time-sequential depth faces that are further augmented with the optical flow features. The augmented features are classified by Generalized Discriminant Analysis (GDA) to make the features more robust and finally, the spatiotemporal features are fed into Hidden Markov Models (HMMs) to train and recognize different facial expressions successfully. The depth information-based proposed facial expression recognition approach is compared to the conventional approaches where the proposed one outperforms others.

Predicting the Sixteen Personality Factors (16PF) of an individual by analyzing facial features

“…Regarding FER systems using other face analysis methods for predicting emotions, we mention the use of Local Directional Pattern (LDP) features [12] extracted from time-sequential depth videos, augmented using optical flows, and classified through Generalized Discriminant Analysis (GDA). The resulted LDP features are then fetched to a chain of HMMs trained to predict the six basic emotions.…”

“…Although at first focused on predicting the emotional state of people [12,13], as Facial Expression Recognition (FER) systems gained momentum and started achieving acceptable prediction accuracy, recent research papers have begun using facial features analysis for more complex tasks, such as tracking and predicting eye gaze [14,15], predicting driver attention for car accident prevention [14,16], predicting stress levels [2,17], diagnosing depression [3], assessing the facial attractiveness of individuals [18], evaluating people's trust [19], and predicting personality traits [4,[20][21][22][23]. All these research studies showed that the face indeed conveys information that can be analyzed to predict different psychological features of an individual.…”