Semantic Deep Face Models

Abstract

Semantic expression synthesis b) Novel identity synthesis c) Subject & expression specific albedo d) 3D retargetting source target e) 2D landmark-based face tracking Figure 1: We propose semantic deep face models-novel neural architectures for modelling and synthesising 3D human faces with the ability to disentangle identity and expression akin to traditional multi-linear models. We demonstrate several applications of our method including (a) semantic expression synthesis, (b) novel identity synthesis (c) generation of expression specific high resolution albedo maps, (d) 3D facial performance retargeting, and (e) 2D landmark based face tracking.

FaceTuneGAN: Face Autoencoder for Convolutional Expression Transfer Using Neural Generative Adversarial Networks

“…Increased realism : while we currently only predict geometry, one could use the data available in FaceScape to learn prediction of displacement maps and textures from the latent codes and generate much more realistic faces. Our network's ability to decouple identity and expression factors could be well-suited to modelling the subtle expressionspecific texture variations, a capability demonstrated by Chandran et al [7].…”

“…Style reconstruction loss: we introduce a novel style reconstruction loss to encourage to preserve the specific style features of the input style mesh in the style latent space: srec ( ) = || ( ) − ( )|| 1 (7) Adversarial loss: our adversarial loss is a conditional loss given by: adv ( , ) = E − log( ( )) +E log(1 − ( )) (8) where (⋅) denotes the discriminator output for the style class of and E[⋅] the mean over the current batch.…”

Cross-Species 3D Face Morphing via Alignment-Aware Controller

“…1) For deforming directly on 3D human faces, Blanz and Vetter (Blanz and Vetter 1999) derived a morphable face model known as 3DMM. Following this framework, many researchers Li et al 2020;Geng, Cao, and Tulyakov 2019;Jiang et al 2019;Ranjan et al 2018;Chandran et al 2020;Chen and Kim 2021;Bailey et al 2020) directly deformed the original 3D human face meshes to have different expressions. In these works, only facial expressions changed after the deformation while the overall shapes were still similar.…”

“…3D face morphing with artificial design has received considerable attention in computer vision community for a long time. Previous methods mainly fall into two categories: 1) Deforming a 3D face towards different expressions and shapes Li et al 2020;Geng, Cao, and Tulyakov 2019;Jiang et al 2019;Ranjan et al 2018;Chandran et al 2020), and 2) Transferring deformations of source faces to control a new target avatar (e.g., face retargeting or reenactment) (Thies et al 2016;Chaudhuri, Vesdapunt, and Wang 2019;Ouzounis, Kilias, and Mousas 2017;Ribera et al 2017;Gao et al 2020;Yao et al 2020;Le and Deng 2017). These works do not consider directly morphing 3D human faces to different structures such as human-to-animal morphing where the structures and feature details of source human faces and target animal faces are quite different.…”

Structure-aware Editable Morphable Model for 3D Facial Detail Animation and Manipulation

“…Furthermore, [75,16,13,82] model the joint distribution of identity and expression by constructing a multilinear model. After statistical analysis of the facial geometry, morphable models can generate 3D faces from a compact latent space and perform expression editing [3,41,37,17] through latent space editing.…”