Towards Fast, Accurate and Stable 3D Dense Face Alignment

Guo, Jianzhu; Zhu, Xiangyu; Yang, Yang; Yang, Fan; Lei, Zhen; Li, Stan Z.

doi:10.1007/978-3-030-58529-7_10

Cited by 291 publications

(318 citation statements)

References 47 publications

Supporting

Mentioning

316

Contrasting

Order By: Relevance

“…Other approaches have used single views with 2D keypoint annotations [9], [25], [55], [56] or object masks [23], [25], [26], [29]. For objects such as human bodies and human faces, some methods [16], [17], [18], [20], [21], [29], [57], [58], [59], [60], [61] have learn to reconstruct from raw images, but starting from the knowledge of a predefined shape model, such as SMPL [62] or Basel [11], or shape templates. These prior models are constructed using specialized hardware and/or other forms of supervision, which are often difficult to obtain for deformable objects in the wild, such as animals, and also limited in details of the shape.…”

Section: Category-specific Reconstructionmentioning

confidence: 99%

See 1 more Smart Citation

Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wild

Wu¹,

Rupprecht²,

Vedaldi³

2021

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

We propose a method to learn 3D deformable object categories from raw single-view images, without external supervision. The method is based on an autoencoder that factors each input image into depth, albedo, viewpoint and illumination. In order to disentangle these components without supervision, we use the fact that many object categories have, at least approximately, a symmetric structure. We show that reasoning about illumination allows us to exploit the underlying object symmetry even if the appearance is not symmetric due to shading. Furthermore, we model objects that are probably, but not certainly, symmetric by predicting a symmetry probability map, learned end-to-end with the other components of the model. Our experiments show that this method can recover very accurately the 3D shape of human faces, cat faces and cars from single-view images, without any supervision or a prior shape model. On benchmarks, we demonstrate superior accuracy compared to another method that uses supervision at the level of 2D image correspondences.

show abstract

Section: Category-specific Reconstructionmentioning

confidence: 99%

“…We also evaluate the reconstructed 3D meshes and compare the performance with several recent 3DMM-based reconstruction methods [21], [57], [58], [59], [60], [61] on two 3D face reconstruction benchmarks [21], [91].…”

Section: D Face Reconstruction Benchmarksmentioning

confidence: 99%

Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wild

Wu¹,

Rupprecht²,

Vedaldi³

2021

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

show abstract

“…We directly minimise parameters in an analysis-by-synthesis manner as shown in Figure 1. We place a Gaussian kernel on each projected vertexx j calculated from (12), and obtain softened semantic labelṔ l of lth label on image coordinate z:…”

Section: Analysis-by-synthesismentioning

confidence: 99%

Shape from semantic segmentation via the geometric Rényi divergence

Koizumi

Smith

2021

2021 IEEE Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

show abstract

“…3DMMs have the strength of controllable parameters and free rotation [8,6]. However, they usually [10,11] require landmarks for registration, and their parametric nature leads to a weakening of personal talking style. 2DFLs has the advantage of fast and accurate extraction, which also represent the most original facial shapes.…”

Section: Introductionmentioning

confidence: 99%

Learning Pose-Adaptive Lip Sync with Cascaded Temporal Convolutional Network

Zheng¹,

Song²,

Ji³

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Speech-driven lip sync has become a promising technique for generating and editing talking-head videos. These studies mainly use 3D morphable models or 2D facial landmarks as the intermediate face representations. However, 2D-based methods have been stagnant recently due to their inability to handle out-of-plane rotations, even though the 2D landmarks have the advantage of fast and accurate extraction. In this paper, we design a cascaded temporal convolutional network to successively generate mouth shapes and corresponding jawlines based on audio signals and template headposes. Instead of explicitly calibrating the rotation between the predicted mouth and the template face, we employ neural networks to learn the pose-adaptive mapping implicitly. We also propose an image-to-image translation-based neural rendering method for producing high-resolution and photo-realistic videos. Experiments show our solution improves both the mapping accuracy and visual performance than baselines. This work could benefit many real-world applications like virtual anchors, telepresence, and conversational agents.

show abstract

Towards Fast, Accurate and Stable 3D Dense Face Alignment

Cited by 291 publications

References 47 publications

Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wild

Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wild

Shape from semantic segmentation via the geometric Rényi divergence

Learning Pose-Adaptive Lip Sync with Cascaded Temporal Convolutional Network

Contact Info

Product

Resources

About