Personalised face neutralisation based on subspace bilinear regression

Abstract

Expression face neutralisation helps to improve the performance of expressive face recognition with one single neutral sample in gallery per subject. For learning-based expression neutralisation, the virtual neutral face totally relies on training samples, which removes person-specific characters from the neutralised face. Bilinear kernel rank reduced regression (BKRRR) algorithm is designed in a virtual subspace to simultaneously and efficiently generate both virtual expressive and neutral images from training samples. An expression mask is then established using grey and gradient differences of the two images. The test expression image is transformed to neutral template by piece-wise affine warp (PAW). Using the virtual BKRRR neutral image as source, the PAW image as destination and the area covered by expression mask as clone area, an image fusion strategy based on Poisson equation is then designed, which achieves virtual neutralised face image with personspecific characters preserved. From experiments on the CMU Multi-PIE databases, it could be observed that the neutral faces synthesised by the proposed method could effectively approximate the real ground truth expressive faces, and greatly improve the performance of classic face recognition algorithms on expression variant problems.

Personalised‐face neutralisation using best‐matched face shape with a neutral‐face database

“…In accordance with our challenge to use only neutral-face databases to identify facial expressions, only the database of neutral faces is used to train and test with other facial expressions. In the experiments, five expressions, namely: disgust, scream, smile, squint and surprise are applied in the same way as the conventional methods [35] trained with facial expressions. All face images in the database are first cropped using landmark points as a guideline.…”

“…For neutralising facial expressions, the conventional method [35], which uses parameters of expression and neutral faces transformed in virtual subspaces to remove the facial expressions, utilises texture and wrinkle as features for face classification and it requires a number of facial‐expression databases. Based on the changes in facial expressions, facial features such as texture, wrinkle and shape become physically modified and all these features should be considered in the classification step by comparing them with the facial‐expression databases.…”

“…The conventional methods [32,35] use fixed-face template for warping textures. We have examined and analysed the face templates of these conventional methods by finding the Euclidean distance error with its ground-truth neutral shape on CMU Multi-PIE.…”

“…Their work involved a high computational time in the warping process. Chen et al [35] presented a method of personalised‐face neutralisation using bilinear kernel rank reduced regression. This method determines a neutral template for initially warping all expression and neutral faces in a database, and performs matching with input face based on subspace bilinear regression, which requires expression‐face databases, and the fixed neutral template may not be suitable for all face warping.…”

Neutral expression synthesis using kernel active shape model

“…Synthesizing facial expression is a challenging task and would require high computational to generate a realistic facial expression. According to [8], the performance of expressive face recognition with a single neutral sample in gallery per subject could be enhance by synthesizing neutral facial expression. Other than the synthesis of facial expression, 3D face model has also been used to synthesize facial aging [9].…”