Random Gabor based templates for facial expression recognition in images with facial occlusion

Abstract

Robust facial expression recognition (FER) under occluded face conditions is challenging. It requires robust algorithms of feature extraction and investigations into the effects of different types of occlusion on the recognition performance to gain insight. Previous FER studies in this area have been limited. They have spanned recovery strategies for loss of local texture information and testing limited to only a few types of occlusion and predominantly a matched train-test strategy. This paper proposes a robust approach that employs a Monte Carlo algorithm to extract a set of Gabor based part-face templates from gallery images and converts these templates into template match distance features. The resulting feature vectors are robust to occlusion because occluded parts are covered by some but not all of the random templates. The method is evaluated using facial images with occluded regions around the eyes and the mouth, randomly placed occlusion patches of different sizes, and near-realistic occlusion of eyes with clear and solid glasses. Both matched and mis-matched train and test strategies are adopted to analyze the effects of such occlusion. Overall recognition performance and the performance for each facial expression are investigated. Experimental results on the Cohn-Kanade and JAFFE databases demonstrate the high robustness and fast processing speed of our approach, and provide useful insight into the effects of occlusion on FER. The results on the parameter sensitivity demonstrate a certain level of robustness of the approach to changes in the orientation and scale of Gabor filters, the size of templates, and occlusions ratios. Performance comparisons with previous approaches show that the proposed method is more robust to occlusion with lower reductions in accuracy from occlusion of eyes or mouth.

Facial Expression Recognition Based on Local Fourier Coefficients and Facial Fourier Descriptors

“…Methods [6], [25], [28] and [35] proposed different approaches for solving this problem. Our potential solution consists of excluding the occluded facial region in the feature extraction process.…”

Facial Expression Analysis under Partial Occlusion

“…The system showed robustness to changes in parameters of Gabor filters and template sizes. The work [Zhang, et al 2011], [Zhang, et al 2014b] adopted features extracted from a large number of random local patches to overcome facial occlusion, while another commonly used way is to extract features from a set of facial sub-regions. Guo and Ruan [2011] used a sum function to combine Local Binary Covariance Matrices (LBCM) features from nine equally sized facial sub-regions.…”

“…LGBP [Azmi and Yegane 2012] 95.6 89.1(-6.5) 97.8(+2.2) 89.1(-6.5) 95(-0.6) --Bayesian [Miyakoshi and Kato 2011] 78.1 78.1(-0) 56.3(-21.8) ----SA CK Gabor [Kotsia, et al 2008] 93 88.5(-4.5) 87.4(-5.6) ----DNMF [Kotsia, et al 2008] 90.4 89.3(-1.1) 88.7(-1.7) ----SVM [Kotsia, et al 2008] 89.5 86.7(-2.8) 82.8(-6.7) ----GSNMF [Zhi, et al 2011] 95. [Kotsia, et al 2008] 86.7 84.2(-2.5) 82.9(-3.8) ----SVM [Kotsia, et al 2008] 91.4 88.4(-3) 86.7(-4.7) ----CFDWL [Huang, et al 2012] 93.2 93(-0.2) 79.1(-14.1) -73.5(-19.7) -86.8(-6.4) Gabor [Zhang, et al 2014b] 95.3 95.1(-0.2) 90.8(-4.5) ---75 (-20.3) MCC [Buciu, et al 2005] 93.6 87.2(-6.4) 92.3(-1.3) ----PCA [Towner and Slater 2007] 75, 85 --70(-5) 82(-3) --SRC [Ouyang, et al 2013] 97.7 72.4(-25.3) -----…”

Multi-scale and Multi-orientation Face Recognition using Voting based Extreme Learning Machine

“…Pose: The images of a face vary due to the relative camera-face pose (frontal, 45 degree, profile, upside down), and some facial features changes due to the partial or whole occlusion of the image [7,3].…”