Large Scale 3D Morphable Models

Abstract

We present large scale facial model (LSFM)-a 3D Morphable Model (3DMM) automatically constructed from 9663 distinct facial identities. To the best of our knowledge LSFM is the largest-scale Morphable Model ever constructed, containing statistical information from a huge variety of the human population. To build such a large model we introduce a novel fully automated and robust Morphable Model construction pipeline, informed by an evaluation of state-of-the-art dense correspondence techniques. The dataset that LSFM is trained on includes rich demographic information about each subject, allowing for the construction of not only a global 3DMM model but also models tailored for specific age, gender or ethnicity groups. We utilize the proposed model to perform age classification from 3D shape alone and to reconstruct noisy out-of-sample data in the low-dimensional model space. Furthermore, we perform a systematic analysis of the constructed 3DMM models that showcases their quality and descriptive power. reveal that the proposed 3DMM achieves state-of-the-art results, outperforming existing models by a large margin. Finally, for the benefit of the research community, we make publicly available the source code of the proposed automatic 3DMM construction pipeline, as well as the constructed global 3DMM and a variety of bespoke models tailored by age, gender and ethnicity.

Modelling of Orthogonal Craniofacial Profiles

“…The corresponding points are known on a template mesh, which is then morphed onto the training scan using underconstrained per-vertex affine transformations, which are constrained by regularisation across neighbouring points [5]. Recently, a similar morphing strategy was used to build a facial 3DMM from a much larger dataset (almost 10,000 faces) [6], although there is no cranial shape information in the model.…”

Text-based editing of talking-head video

“…There is a large body of work on reconstructing facial geometry and appearance from a single image using optimization methods [Fyffe et al 2014;Garrido et al 2016;Ichim et al 2015;Kemelmacher-Shlizerman 2013;Roth et al 2017;Shi et al 2014;Suwajanakorn et al 2017;Thies et al 2016]. Many of these techniques employ a parametric face model [Blanz et al 2004;Blanz and Vetter 1999;Booth et al 2018] as a prior to better constrain the reconstruction problem. Recently, deep learning-based approaches have been proposed that train a convolutional network Given an input talking-head video and a transcript, we perform text-based editing.…”

Self-Supervised Adaptation of High-Fidelity Face Models for Monocular Performance Tracking

“…Individual faces are combined into a single 3DMM [1] by computing dense correspondences based on optical flow in conjunction with the shape and texture priors in a linear subspace. Large-scale face scans (more than 10,000 people) from diverse population enables modeling of accurate distributions of faces [3,2]. With the aid of multi-camera systems and deep neural networks, the limitation of the linear models can be overcome using DAMs [13] that predicts high quality geometry and texture.…”