Facial expression recognition in dynamic sequences: An integrated approach

Abstract

Automatic facial expression analysis aims to analyse human facial expressions and classify them into discrete categories. Methods based on existing work are reliant on extracting information from video sequences and employ either some form of subjective thresholding of dynamic information or attempt to identify the particular individual frames in which the expected behaviour occurs. These methods are inefficient as they require either additional subjective information, tedious manual work or fail to take advantage of the information contained in the dynamic signature from facial movements for the task of expression recognition.In this paper, a novel framework is proposed for automatic facial expression analysis which extracts salient information from video sequences but does not rely on any subjective preprocessing or additional user-supplied information to select frames with peak expressions. The experimental framework demonstrates that the proposed method outperforms static expression recognition systems in terms of recognition rate. The approach does not rely on action units (AUs) and therefore, eliminates errors which are otherwise propagated to the final result due to incorrect initial identification of AUs. The proposed framework explores a parametric space of over 300 dimensions and is tested with six state-of-the-art machine learning techniques. Such robust and extensive experimentation provides an important foundation for the assessment of the performance for future work. A further contribution of the paper is offered in the form of a user study. This was conducted in order to investigate the correlation between human cognitive systems and the proposed framework for the understanding of human emotion classification and the reliability of public databases.

A dynamic framework based on local Zernike moment and motion history image for facial expression recognition

“…Thus, we can also conclude that the combination of two dynamic features improves the recognition rate. We also conducted an experiment on the MMI dataset, comparing the proposed framework with the method that uses LBP and SVM [37], and the methods in [45] and [44] that are evaluated using the same classification strategy of 10-fold cross-validation. The average recognition rates are shown in Table 7.…”

“…The table shows that the proposed framework outperforms all the other five methods. The result for LBP was obtained by using different samples to those used in [37], and using the same strategy of classification introduced in [45] which is also used in [44] and the proposed method. Although CK+ and MMI are two of the most widely used datasets for evaluating facial expression recognition methods, they are both collected in a strict controlled settings with near frontal poses, consistent illumination and posed expressions.…”

A spatial-temporal framework based on histogram of gradients and optical flow for facial expression recognition in video sequences

“…[27], Fang et al [31], AAM [31], and ASM [31]. For the experiments, we performed 10-fold cross-validation.…”

Development of a Computer System for Identity Authentication Using Artificial Neural Networks

“…A key example reflecting the basic principles of these technologies is the automatic detection of human faces based on the electronic video picture. It is necessary to fight terrorism and crime (Iwama et al, 2013), for the general control of migration (Sanchez del Rio et al, 2016), for personal identification in bank transactions via electronic networks and a number of related tasks, where the price of mistaken identity is very high (Didimo et al, 2014;Liao et al, 2015;Fang et al,2014).…”