A high fidelity synthetic face framework for computer vision

Abstract

Analysis of faces is one of the core applications of computer vision, with tasks ranging from landmark alignment, head pose estimation, expression recognition, and face recognition among others. However, building reliable methods requires time-consuming data collection and often even more time-consuming manual anotation, which can be unreliable. In our work we propose synthesizing such facial data, including ground truth annotations that would be almost impossible to acquire through manual annotation at the consistency and scale possible through use of synthetic data. We use a parametric face model together with hand crafted assets which enable us to generate training data with unprecedented quality and diversity (varying shape, texture, expression, pose, lighting, and hair).

Synthetic Data for Multi-Parameter Camera-Based Physiological Sensing

“…There is a long history of the use of synthetic data in training and evaluating computer vision systems [23], [24], [25], [26], [27], [28], [29], [30], [7], [5], [8], [4], [6]. Synthetics have been employed extensively in models for face and body analysis specifically [31], [7], [28], [29], [32]. But the same is not true for camera-based physiological sensing.…”

GAN-Based Face Attribute Editing

“…Instead of 3DMM, ConfigNet [35] creates a set of face images with known semantic parameters and finer textural details following [165], and trains a generator G on Synth-Face to model the mapping between semantic parameters and face images. An encoder network is then trained to embed real images into the parameter space of G, which enables real image manipulation.…”