Towards High Fidelity Monocular Face Reconstruction with Rich Reflectance using Self-supervised Learning and Ray Tracing

Dib, Abdallah; Thébault, Cédric; Ahn, Jeong Joon; Gosselin, Philippe-Henri; Theobalt, Christian; Chevallier, Louis

doi:10.1109/iccv48922.2021.01258

Cited by 28 publications

(9 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A conclusion can be drawn that reconstructing 3D face models by image-to-image methods is less effective. The performance of our reconstructed 3D shapes is slightly worse than that of Dib et al [46], 3DDFA-V2 [8], Deng et al [26], and RingNet [27], where the median error differs by only approximately 0.03 mm to 0.08 mm, and the cumulative error is slightly higher but close to these comparative approaches. The main reason is that most of them chose to regress more 3DMM parameters to obtain more accurate results, which means more time and memory consuming processes.…”

Section: Evaluation Of 3d Face Reconstructioncontrasting

confidence: 56%

“…The main reason is that most of them chose to regress more 3DMM parameters to obtain more accurate results, which means more time and memory consuming processes. Dib et al [46], Deng et al [26], and RingNet [27] all employed a deep residual network [47] with equal to or more than 50 layers. The network would run slowly or even fail to run on typical CPUs.…”

Section: Evaluation Of 3d Face Reconstructionmentioning

confidence: 99%

See 1 more Smart Citation

Fast 3D face reconstruction from a single image combining attention mechanism and graph convolutional network

et al. 2022

View full text Add to dashboard Cite

In recent years, researchers have made significant contributions to 3D face reconstruction with the rapid development of deep learning. However, learning-based methods often suffer from time and memory consumption. Simply removing network layers hardly solves the problem. In this study, we propose a solution that achieves fast and robust 3D face reconstruction from a single image without the need for accurate 3D data for training. In terms of increasing speed, we use a lightweight network as a facial feature extractor. As a result, our method reduces the reliance on graphics processing units, allowing fast inference on central processing units alone. To maintain robustness, we combine an attention mechanism and a graph convolutional network in parameter regression to concentrate on facial details. We experiment with different combinations of three loss functions to obtain the best results. In comparative experiments, we evaluate the performance of the proposed method and state-of-the-art methods on 3D face reconstruction and sparse face alignment, respectively. Experiments on a variety of datasets validate the effectiveness of our method.

show abstract

Section: Evaluation Of 3d Face Reconstructioncontrasting

confidence: 56%

Section: Evaluation Of 3d Face Reconstructionmentioning

confidence: 99%

Fast 3D face reconstruction from a single image combining attention mechanism and graph convolutional network

et al. 2022

View full text Add to dashboard Cite

show abstract

“…This includes material acquisition in‐the‐wild [RRFG17,NLG22,LLK*22], or using flash‐ and no‐flash image pairs [CWS*20, PCF05], which are also applicable to human faces but limited to a macro scale. In a different thread, many neural networks have been proposed to estimate the 3D geometry directly from one or several 2D face images [TZK*17,GPKo19,DYX*19, GZY*20,FFBB21,DTA*21,WBH*22]. While these approaches are attractive in their ease of usability, and many can achieve real‐time reconstruction, they target low resolution base‐level geometry that contains neither macro nor micro details.…”

Section: Related Workmentioning

confidence: 99%

Graph‐Based Synthesis for Skin Micro Wrinkles

Weiss,

Moulin,

Chandran

et al. 2023

Computer Graphics Forum

View full text Add to dashboard Cite

We present a novel graph‐based simulation approach for generating micro wrinkle geometry on human skin, which can easily scale up to the micro‐meter range and millions of wrinkles. The simulation first samples pores on the skin and treats them as nodes in a graph. These nodes are then connected and the resulting edges become candidate wrinkles. An iterative optimization inspired by pedestrian trail formation is then used to assign weights to those edges, i.e., to carve out the wrinkles. Finally, we convert the graph to a detailed skin displacement map using novel shape functions implemented in graphics shaders. Our simulation and displacement map creation steps expose fine controls over the appearance at real‐time framerates suitable for interactive exploration and design. We demonstrate the effectiveness of the generated wrinkles by enhancing state‐of‐art 3D reconstructions of real human subjects with simulated micro wrinkles, and furthermore propose an artist‐driven design flow for adding micro wrinkles to fictional characters.

show abstract

“…In an alternative approach, [12,13] employ a generative adversarial network for 3D face fitting. Recent works have also employed 3DMMs to estimate 3D geometry and spatially varying surface properties, such as diffuse and specular albedos, along with global illumination properties [8,8,10].…”

Section: Related Workmentioning

confidence: 99%

SIRA: Relightable Avatars from a Single Image

Pol¹,

Ramon²,

García³

et al. 2023

2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

Recent advancements in learning techniques that employ coordinate-based neural representations have yielded remarkable results in multi-view 3D reconstruction tasks. However, these approaches often require a substantial number of input views (typically several tens) and computationally intensive optimization procedures to achieve their effectiveness. In this paper, we address these limitations specifically for the problem of few-shot full 3D head reconstruction. We accomplish this by incorporating a probabilistic shape and appearance prior into coordinate-based representations, enabling faster convergence and improved generalization when working with only a few input images (even as low as a single image). During testing, we leverage this prior to guide the fitting process of a signed distance function using a differentiable renderer. By incorporating the statistical prior alongside parallelizable ray tracing and dynamic caching strategies, we achieve an efficient and accurate approach to few-shot full 3D head reconstruction. Moreover, we extend the H3DS dataset, which now comprises 60 high-resolution 3D full head scans and their corresponding posed images and masks, which we use for evaluation purposes. By leveraging this dataset, we demonstrate the remarkable capabilities of our approach in achieving state-of-the-art results in geometry reconstruction while being an order of magnitude faster than previous approaches.

show abstract

Towards High Fidelity Monocular Face Reconstruction with Rich Reflectance using Self-supervised Learning and Ray Tracing

Cited by 28 publications

References 44 publications

Fast 3D face reconstruction from a single image combining attention mechanism and graph convolutional network

Fast 3D face reconstruction from a single image combining attention mechanism and graph convolutional network

Graph‐Based Synthesis for Skin Micro Wrinkles

SIRA: Relightable Avatars from a Single Image

Contact Info

Product

Resources

About