Learning Formation of Physically-Based Face Attributes

Li, Ruilong; Bladin, Karl; Zhao, Yajie; Chinara, Chinmay; Ingraham, Owen; Xiang, Pengda; Ren, Xinglei; Prasad, Pratusha Bhuvana; Bhamidipati, Kishore; Xing, Junliang; Li, Hao

doi:10.1109/cvpr42600.2020.00347

Cited by 73 publications

(48 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The more recent work of Li et. al [24] is closest in spirit to our work. Though our methods seem similar at the onset, there are a few important differences.…”

Section: Related Workmentioning

confidence: 61%

“…The first is that though we decouple identity and expression in the network's latent space, our joint decoder can model identity specific expression deformations which [24] can not. Second, as we describe in Section 3.2, the manner in which we use dynamic facial performances for training readily makes our method applicable to retarget and reconstruct performance from videos, and addresses another limitation of [24]. Another interesting contribution in neural semantic face modelling is the work of Bailey et.…”

Section: Related Workmentioning

confidence: 99%

“…As we shall see in more detail in Section 2, recent methods have begun to investigate nonlinear face models using neural networks [28,1,14,20,16,24,3], which can, to some degree, overcome the limitations of linear models. Unfortunately, some of these approaches have thus far sacrificed the human interpretable nature of multi-linear models, as one typically loses semantics when moving to a latent space learned by a deep network.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Semantic Deep Face Models

Chandran

Bradley

Groß

et al. 2020

2020 International Conference on 3D Vision (3DV)

View full text Add to dashboard Cite

Semantic expression synthesis b) Novel identity synthesis c) Subject & expression specific albedo d) 3D retargetting source target e) 2D landmark-based face tracking Figure 1: We propose semantic deep face models-novel neural architectures for modelling and synthesising 3D human faces with the ability to disentangle identity and expression akin to traditional multi-linear models. We demonstrate several applications of our method including (a) semantic expression synthesis, (b) novel identity synthesis (c) generation of expression specific high resolution albedo maps, (d) 3D facial performance retargeting, and (e) 2D landmark based face tracking.

show abstract

“…The more recent work of Li et. al [24] is closest in spirit to our work. Though our methods seem similar at the onset, there are a few important differences.…”

Section: Related Workmentioning

confidence: 61%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Semantic Deep Face Models

Chandran

Bradley

Groß

et al. 2020

2020 International Conference on 3D Vision (3DV)

View full text Add to dashboard Cite

show abstract

“…Instead of representing facial shapes and appearances as a linear combination of basis vectors, these models are formulated implicitly as decoders using neural networks where the 3D faces are generated directly from latent vectors. Some of these methods use fully connected layers or 2D convolutions in image space [64,4,22,63,45], while others use decoders in the mesh domain to represent local geome-tries [49,54,19,26,50,43,47]. With the help of differentiable renderers [61,27,55], several methods [64,63,43] have demonstrated high-fidelity 3D face reconstructions using non-linear morphable face models using fully unsupervised or weakly supervised learning, which is possible using massive amounts of images in the wild.…”

Section: Related Workmentioning

confidence: 99%

Normalized Avatar Synthesis Using StyleGAN and Perceptual Refinement

Luo¹,

Nagano²,

Kung³

et al. 2021

Preprint

View full text Add to dashboard Cite

We introduce a highly robust GAN-based framework for digitizing a normalized 3D avatar of a person from a single unconstrained photo. While the input image can be of a smiling person or taken in extreme lighting conditions, our method can reliably produce a high-quality textured model of a person's face in neutral expression and skin textures under diffuse lighting condition. Cutting-edge 3D face reconstruction methods use non-linear morphable face models combined with GAN-based decoders to capture the likeness and details of a person but fail to produce neutral head models with unshaded albedo textures which is critical for creating relightable and animation-friendly avatars for integration in virtual environments. The key challenges for existing methods to work is the lack of training and ground truth data containing normalized 3D faces. We propose a two-stage approach to address this problem. First, we adopt a highly robust normalized 3D face generator by embedding a non-linear morphable face model into a StyleGAN2 network. This allows us to generate detailed but normalized facial assets. This inference is then followed by a perceptual refinement step that uses the generated assets as regularization to cope with the limited available training samples of normalized faces. We further introduce a Normalized Face Dataset, which consists of a combination photogrammetry scans, carefully selected photographs, and generated fake people with neutral expressions in diffuse lighting conditions. While our prepared dataset contains two orders of magnitude less subjects than cutting edge GAN-based 3D facial reconstruction methods, we show that it is possible to produce high-quality normalized face models for very challenging unconstrained input images, and demonstrate superior performance to the current state-of-the-art.

show abstract

“…The proliferation of deep generative models enabled realistic face creation and manipulation techniques such as reenactment Thies et al 2016;Wang et al 2018], attribute or domain manipulation He et al 2017;, expression molding [Ding et al 2018], and inpainting [Li et al 2017b]. In particular, physically grounded models [Li et al 2020a[Li et al , 2017aSaito et al 2017] generate realistic humans by learning facial parameters. Although there is a few techniques focusing on gaze reenactment [Ganin et al 2016;, artifacts remain when head is not front-facing and inconsistencies exist between pose and perspective.…”

Section: Parametric Face Synthesismentioning

confidence: 99%