Head Reconstruction from Internet Photos

Liang, Shu; Shapiro, Linda G.; Kemelmacher-Shlizerman, Ira

doi:10.1007/978-3-319-46475-6_23

Cited by 21 publications

(13 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Kemelmacher-Shlizerman [30] learns a 3DMM from a large photo-collection of people, grouped into a fixed set of semantic labels. Also, Liang et al [37] leverage multi-view person-specific photo-collections to reconstruct the full head. In a different line of research, Thies et al [61] fit a coarse parametric model to user-selected views to recover personalized face shape and albedo.…”

Section: Related Workmentioning

confidence: 99%

FML: Face Model Learning From Videos

Tewari

Bernard

Garrido

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

165

149

View full text Add to dashboard Cite

Figure 1. We propose multi-frame self-supervised training of a deep network based on in-the-wild video data for jointly learning a face model and 3D face reconstruction. Our approach successfully disentangles facial shape, appearance, expression, and scene illumination. AbstractMonocular image-based 3D reconstruction of faces is a long-standing problem in computer vision. Since image data is a 2D projection of a 3D face, the resulting depth ambiguity makes the problem ill-posed. Most existing methods rely on data-driven priors that are built from limited 3D face scans. In contrast, we propose multi-frame video-based self-supervised training of a deep network that (i) learns a face identity model both in shape and appearance while (ii) jointly learning to reconstruct 3D faces. Our face model is learned using only corpora of in-the-wild video clips collected from the Internet. This virtually endless source of training data enables learning of a highly general 3D face model. In order to achieve this, we propose a novel multiframe consistency loss that ensures consistent shape and appearance across multiple frames of a subject's face, thus minimizing depth ambiguity. At test time we can use an arbitrary number of frames, so that we can perform both monocular as well as multi-frame reconstruction.

show abstract

Section: Related Workmentioning

confidence: 99%

FML: Face Model Learning From Videos

Tewari

Bernard

Garrido

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

165

149

View full text Add to dashboard Cite

show abstract

“…As an application experiment, we outline a methodology that enables us to reconstruct the entire head shape from unconstrained single images. We strictly utilize only one view/pose for head reconstruction in contrast to [21] where multiple images were utilized with photometric constraints. We achieve this by regressing from a latent space that represents the 3D face and ear shape to the latent space of the full head models constructed by the proposed methodologies.…”

Section: Head Reconstruction From Single Imagesmentioning

confidence: 99%

Combining 3D Morphable Models: A Large Scale Face-And-Head Model

Ploumpis

Wang

Pears

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

framework to blend covariance matrices from multiple models. Second, as an example application of our approach, we build a new face-and-head shape model that combines the variability and facial detail of the LSFM with the full head modelling of the LYHM. The resulting combined shape model achieves state-of-the-art performance and outperforms existing head models by a large margin. Finally, as an application experiment, we reconstruct full head representations from single, unconstrained images by utilizing our proposed large-scale model in conjunction with the Face-Warehouse blendshapes for handling expressions.

show abstract

“…As an independent application, specific frontalization techniques have also been proposed [9]. Another line of work pertains to 3D face reconstruction from photo collections [29,18,42] or a single image [19,50,40], where the latter have been successfully used for face normalization prior to recognition. While most of the methods apply the framework of aligning 3DMM with the 2D face landmarks [47,46,25] and conduct further refinement.…”

Section: Related Workmentioning

confidence: 99%

Reconstruction-Based Disentanglement for Pose-Invariant Face Recognition

Peng

Sohn

et al. 2017

2017 IEEE International Conference on Computer Vision (ICCV)

143

View full text Add to dashboard Cite

Deep neural networks (DNNs) trained on large-scale datasets have recently achieved impressive improvements in face recognition. But a persistent challenge remains to develop methods capable of handling large pose variations that are relatively under-represented in training data. This paper presents a method for learning a feature representation that is invariant to pose, without requiring extensive pose coverage in training data. We first propose to generate non-frontal views from a single frontal face, in order to increase the diversity of training data while preserving accurate facial details that are critical for identity discrimination. Our next contribution is to seek a rich embedding that encodes identity features, as well as non-identity ones such as pose and landmark locations. Finally, we propose a new feature reconstruction metric learning to explicitly disentangle identity and pose, by demanding alignment between the feature reconstructions through various combinations of identity and pose features, which is obtained from two images of the same subject. Experiments on both controlled and in-the-wild face datasets, such as MultiPIE, 300WLP and the profile view database CFP, show that our method consistently outperforms the state-of-the-art, especially on images with large head pose variations. 1

show abstract

Head Reconstruction from Internet Photos

Cited by 21 publications

References 39 publications

FML: Face Model Learning From Videos

FML: Face Model Learning From Videos

Combining 3D Morphable Models: A Large Scale Face-And-Head Model

Reconstruction-Based Disentanglement for Pose-Invariant Face Recognition

Contact Info

Product

Resources

About