A Coupled Statistical Model for Face Shape Recovery From Brightness Images

Abstract

We focus on the problem of developing a coupled statistical model that can be used to recover facial shape from brightness images of faces. We study three alternative representations for facial shape. These are the surface height function, the surface gradient, and a Fourier basis representation. We jointly capture variations in intensity and the surface shape representations using a coupled statistical model. The model is constructed by performing principal components analysis on sets of parameters describing the contents of the intensity images and the facial shape representations. By fitting the coupled model to intensity data, facial shape is implicitly recovered from the shape parameters. Experiments show that the coupled model is able to generate accurate shape from out-of-training-sample intensity images.

3D shape regression for real-time facial animation

“…PCA shape models have commonly been fit to face images to recover facial shape [Blanz and Vetter 1999;Matthews et al 2007;Cootes et al 2012]. PCA has also been used to model other facial shape descriptors, such as surface height and surface gradient fields [Castelan et al 2007]. While PCA face models provide a compact representation and computational simplicity, these generic parametric models have a limited capacity to represent the particular detailed shape variations of an individual's facial expressions, due to the limited number of PCA dimensions that can be used in practice.…”

Recovering 3D facial shape via coupled 2D/3D space learning

“…However, the best fit coefficients in the intensity space are not necessarily the best fit coefficients in the shape space. To solve this problem, they performed similar coupled statistical model on the coefficients obtained from the original coupled model [14]. This approach is similar to the active appearance model proposed by Cootes et al [15].…”

3D facial shape estimation from a single image under arbitrary pose and illumination.

“…Notice that the results are very difficult to differentiate from each other. Castelan's methods [1] [14] have an additional major difference in that they make use only of shape and texture models. There is no spherical harmonics projection (SHP) model that can deal with illumination in the input image.…”

“…The morphable model framework of [35] estimates the shape and texture coefficients from an input 2D image, together with other scene parameters, using an optimization method based on stochastic gra- Castelan et al [1] developed a coupled statistical model, which is a variant of the combined AAM [36], that can recover 3D shape from intensity images with a frontal pose. The shape and intensity models in Castelan's work is similar to that of the AAM model discussed in the appendix.…”

“…There is no pose invariance in both Castelan [1] and Ahmed et al [2] approaches, i.e., only frontal poses are considered. This dissertation can handle unknown pose with the help of multiple-view geometry [3] concepts.…”