HeadGAN: One-shot Neural Head Synthesis and Editing

Abstract

Reference image (b) Reenactment (c) Reconstruction (self-reenactment) (d) Expression Editing (e) Pose Editing (f) Frontalisation Figure 1: Our proposed HeadGAN method performs reenactment (b), by fully transferring the facial expressions and head pose from a driving frame to a reference image. When the driving and reference identities coincide (c), it can be used for facial video compression and reconstruction. In addition, HeadGAN can be applied to facial expression editing (d), novel view synthesis (e) and face frontalisation (f).

StyleHEAT: One-Shot High-Resolution Editable Talking Face Generation via Pre-trained StyleGAN

“…X2Face [22] 256 Bi-layer [68] 256 FOMM [40] 256 HeadGAN [15] 256 face-Vid2Vid [58] 512 HDTF [72] 512 PC-AVS [74] 224 wav2lip [37] 96 PIRenderer [38] 256 Ours 1024 called W space, which is constructed by mapping a normal distribution to a new distribution via a multi-layer perceptron (MLP).…”

“…NS-PVD [30] extends DVP by a novel target-style preserving recurrent GAN. Recent 3D model-based methods [15,17,18,38] can also do a good job for subject-agnostic face synthesis. HeadGAN [15] pre-processes the 3d mash as input of the network.…”

“…Recent 3D model-based methods [15,17,18,38] can also do a good job for subject-agnostic face synthesis. HeadGAN [15] pre-processes the 3d mash as input of the network. PIRenderer [38] predicts a flow field for feature warping.…”

“…To achieve accurate and intuitive motion control, semantic medium plays an important role in the generation process. Following previous works [15,38], we take advantage of the 3DMM [7] parameters for motion modeling. In 3DMM, the 3D shape 𝑺 of a face can be decoupled as:…”

“…There are many attempts to drive a static portrait with a video or audio from different perspectives in recent literature. A set of methods [15,38,65,72] take the advantage of 3D Morphable Models (3DMMs), a parametric model that decomposes expression, pose, and identity, to transfer facial motions. For the audio-driven case, the audio features are always projected to the parameter space of 3DMM [65,67,70].…”