Markov Chain Monte Carlo for Automated Face Image Analysis

Schönborn, Sandro; Egger, Bernhard; Morel-Forster, Andreas; Vetter, Thomas

doi:10.1007/s11263-016-0967-5

Cited by 58 publications

(74 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…After registration and model-building, we demonstrate the applicability of the model with an inverse rendering application of 2D face images. Unlike the approach by [26], which is used in this work, most methods only recover shape but ignore color and illumination. An overview over current inverse rendering techniques is contained in [26].…”

Section: Related Workmentioning

confidence: 99%

Morphable Face Models - An Open Framework

Gerig¹,

Morel-Forster²,

Blumer³

et al. 2018

2018 13th IEEE International Conference on Automatic Face &Amp; Gesture Recognition (FG 2018)

Self Cite

230

224

View full text Add to dashboard Cite

In this paper, we present a novel open-source pipeline for face registration based on Gaussian processes as well as an application to face image analysis. Non-rigid registration of faces is significant for many applications in computer vision, such as the construction of 3D Morphable face models (3DMMs). Gaussian Process Morphable Models (GPMMs) unify a variety of non-rigid deformation models with B-splines and PCA models as examples. GPMM separate problem specific requirements from the registration algorithm by incorporating domain-specific adaptions as a prior model. The novelties of this paper are the following: (i) We present a strategy and modeling technique for face registration that considers symmetry, multiscale and spatially-varying details. The registration is applied to neutral faces and facial expressions. (ii) We release an open-source software framework for registration and modelbuilding, demonstrated on the publicly available BU3D-FE database. The released pipeline also contains an implementation of an Analysis-by-Synthesis model adaption of 2D face images, tested on the Multi-PIE and LFW database. This enables the community to reproduce, evaluate and compare the individual steps of registration to model-building and 3D/2D model fitting. (iii) Along with the framework release, we publish a new version of the Basel Face Model (BFM-2017) with an improved age distribution and an additional facial expression model.

show abstract

Section: Related Workmentioning

confidence: 99%

Morphable Face Models - An Open Framework

Gerig¹,

Morel-Forster²,

Blumer³

et al. 2018

2018 13th IEEE International Conference on Automatic Face &Amp; Gesture Recognition (FG 2018)

Self Cite

230

224

View full text Add to dashboard Cite

show abstract

“…Face shape and texture are modelled as a linear combination of Principal Component Analyis (PCA) basis vectors computed from exemplar 3D face scans. The Analysis-by-Synthesis method of Schönborn et al [18] uses a MCMC Metropolis Hastings method to re-construct shape, texture, and illumination from a single image. Analysis-by-Synthesis methods require realistic and fast rendering methods.…”

Section: Related Workmentioning

confidence: 99%

“…Aldrian and Smith et al [2] and Schönborn et al [18] use spherical harmonics to approximate the environment map and can produce more realistic illumination settings but lack shadowing. The work of Shahlaei and Blanz [19] uses an illumination cone to combine multiple light sources to approximate the environment map and includes cast shadows in rendering.…”

Section: Related Workmentioning

confidence: 99%

“…Such estimation of illumination respects self-shadowing as an effect of surface-light interaction. For adapting the model to an image, we employ the Markov chain Monte Carlo fitting framework of Schönborn et al [18].…”

Section: Model Fit To An Imagementioning

confidence: 99%

“…2. Model fitting is based on the MCMC method presented in [18]. We use their presented setup but replace illumination estimation with our own shadow-aware version presented in Section 2.4.…”

Section: Reconstruction Experiments On Artificial Datamentioning

confidence: 99%

See 2 more Smart Citations

Efficient Global Illumination for Morphable Models

Schneider¹,

Schönborn

Egger

et al. 2017

2017 IEEE International Conference on Computer Vision (ICCV)

Self Cite

View full text Add to dashboard Cite

We propose an efficient self-shadowing illumination model for Morphable Models. Simulating self-shadowing with ray casting is computationally expensive which makes them impractical in Analysis-by-Synthesis methods for object reconstruction from single images. Therefore, we propose to learn self-shadowing for Morphable Model parameters directly with a linear model. Radiance transfer functions are a powerful way to represent self-shadowing used within the precomputed radiance transfer framework (PRT). We build on PRT to render deforming objects with self-shadowing at interactive frame rates. It can be illuminated efficiently by environment maps represented with spherical harmonics. The result is an efficient global illumination method for Morphable Models, exploiting an approximated radiance transfer. We apply the method to fitting Morphable Model parameters to a single image of a face and demonstrate that considering self-shadowing improves shape reconstruction.

show abstract

3D face modeling from single image based on discrete shape space

Zhang

Liu

et al. 2020

Computer Animation & Virtual

View full text Add to dashboard Cite

In this article, we propose a novel 3D face modeling method which constructs a new 3D face model from a low-dimensional feature space consisted of a large set of blend shapes based on the discrete shape space theory. The details of original face features are completely retained during the modeling process and a large number of new natural faces are constructed by several face samples. The optimization process of our method is independently decoupled for different facial attributes (identity, expression, and head pose), which improves the application flexibility and reduces the probability of it falling into a local optimal situation. The new facial data with new attributes are constructed based on the geodesic path search in discrete shape space with sufficient freedom and accuracy. In experiments and applications based on public databases (Helen, LFW, and CUFS), the modeling results show our method can provide high-quality 3D face model, with enough freedom for face expression editing and natural facial expression animation from a small facial sample set. K E Y W O R D S 3D face modeling, discrete shape space, facial landmarks, geodesic path 1 INTRODUCTION 3D face modeling is an important issue for many applications such as game production, movie making, and cartoon animation creation. The face data can be edited using the reconstruction face model, which is a powerful tool for imaginative creative activities and business applications (DAZ3D, Maya, and Meitu). It is generally difficult for building a high-quality 3D face model from a single image, because the faces in images have different head poses, face scales and the geometric features are incomplete, which is termed "ill-posed." To remove the influence of ill-posed features, the prior knowledge of the 3D face data should be used to guide the modeling process. Simultaneously, to synthesize the 3D face object from an image, the modeling method is required to match the geometric features (facial landmarks or contours) between the 3D face data and 2D facial image. In general, a standard 3D face object (from Facewarehouse or Bospho-rusDB) has more than 10,000 vertices, it cause that the high computational complexity of directly 3D face modeling in original 3D face objects, which it is unattainable. Therefore, we propose a 3D face modeling method which represents 3D face by facial landmarks can quickly and correctly construct 3D face model based on the discrete space in this article.

show abstract

Markov Chain Monte Carlo for Automated Face Image Analysis

Cited by 58 publications

References 44 publications

Morphable Face Models - An Open Framework

Morphable Face Models - An Open Framework

Efficient Global Illumination for Morphable Models

3D face modeling from single image based on discrete shape space

Contact Info

Product

Resources

About