Interactive Sculpting of Digital Faces Using an Anatomical Modeling Paradigm

Abstract

Figure 1: We build a local anatomical model from a corpus of 3D face scans (left) and create a novel anatomically-driven digital face sculpting system where novice users can efficiently create realistic 3D face identities (right).

3D Shape Variational Autoencoder Latent Disentanglement via Mini-Batch Feature Swapping for Bodies and Faces

“…Direct Manipulation Similarly to [17] our method supports direct manipulation of the generated 3D meshes. A user is thus able to select one or multiple vertices, specify their new desired location, and our method automatically generates a new mesh locally deformed to satisfy the user edit.…”

“…These models are common as consumer-level avatar design tools adopted by several videogame engines. Even though they guarantee control over the generation of local features, they are very large models usually built with only a few subjects and are capable of offering only very limited flexibility and expressivity [17]. A widespread approach to overcome these limitations is to rely on linear statistical 3D morphable models (3DMM).…”

“…This technique was pioneered by [5] and further developed and adopted by many researchers [12]. Interestingly, [17] divided the face in different local patches and trained a PCA model for each region in order to control the generation of different facial features. The generation of new faces and interactive face sculpting are then achieved through a constrained optimisation.…”

“…As aforementioned in Sec. 1, with the exception of artistically-created blendshape models and [17], none of the other methods here described allow to control local changes during the generation process because their generative coefficients lack any semantic meaning, are not easily interpretable and are not properly disentangled.…”

“…Currently, the generation procedure is either manually performed by highly skilled artists or it involves semi-automated avatar design tools. Even though these tools greatly simplify the design process, they are usually limited in flexibility because of the intrinsic constraints of the underlying generative models [17]. Blendshapes [28,31,38], 3D morphable mod-els [5,25,33], autoencoders [3,8,16,35], and generative adversarial networks [1,7,15,24] are currently the most used generative models, but they all share one particular issue: the creation of local features is difficult or even impossible.…”