Full Face-and-Head 3D Model With Photorealistic Texture

Abstract

In the recent period, significant progress has been achieved towards reconstructing the 3D face model from face image. With the support of the render engines and sufficient data, the reconstruction results are fine in detail. Nevertheless, the research on the 3D face reconstruction with texture from a single unrestricted face image is imperfect. The rebuild process lacks essential structure and texture information in the profile and the craniofacial region. To address this problem, we present a method of creating a 3D full face-and-head model with photorealistic texture from a single "in-the-wild" face image in this paper. To this end, we introduce a pipeline to integrate the highly-detailed face model into the basic model. Specifically, the basic model was built by multilinear optimization, and the highly-detailed face model which represents the facial features generated by constrained illumination distribution. Additionally, to infer the invisible region texture information corresponding to the input face image, we design an effective architecture with the generative adversarial network (GAN) for panoramic UV texture generation. The final results after UV texture mapping were visualized in the experiment, which demonstrates that the model faithfully recovers the photorealistic details in arbitrary perspective. Furthermore, compared to the state-of-the-art facial modeling techniques and existing commercial solutions, our method takes less input and performs better in surface detail. INDEX TERMS 3D face reconstruction, full face-and-head model, 3D Morphable Model, UV texture, Generative Adversarial Networks.

Evaluation metrics for intelligent generation of graphical game assets: a systematic survey-based framework

“…Alternatively, some similarity metrics are specific to 2D assets. For example, peak signal-to-noise ratio (PSNR) can be used to determine the strength of noise within an image, and therefore can be a measure of visual quality [47], [92], [93]. Structural similarity (SSIM) measures the perceptual quality based on high-level structure, comparing the output to a ground-truth image [94].…”

“…Visual balance [14] Objective similarity SSIM [92], [129] [47], [93], [130], [17], [19], [95], [131] [38] PSNR [92] [47], [93], [130], [95], [131] [15]…”

Realistic Facial Three-Dimensional Reconstruction from CT Images and 2D Photographic Images for Surgical-Orthognathic Planning

“…Recently, significant progress has been achieved regarding the 3D reconstruction of facial models from a single portrait image. According to current scientific evidence, it has been concluded that this method has a superior performance compared to the others, since it requires a smaller number of resources (one portrait photo-graph) and presents greater surface detail [19]. However, CBCT reconstruction may include artefacts caused by restorations or orthodontic appliances or defects in the clear representation of facial patient features (e.g., face color), which affect the optimal representation of the 3D facial image.…”