Beyond 3DMM: Learning to Capture High-Fidelity 3D Face Shape

Abstract

3D Morphable Model (3DMM) fitting has widely benefited face analysis due to its strong 3D priori. However, previous reconstructed 3D faces suffer from degraded visual verisimilitude due to the loss of fine-grained geometry, which is attributed to insufficient ground-truth 3D shapes, unreliable training strategies and limited representation power of 3DMM. To alleviate this issue, this paper proposes a complete solution to capture the personalized shape so that the reconstructed shape looks identical to the corresponding person. Specifically, given a 2D image as the input, we virtually render the image in several calibrated views to normalize pose variations while preserving the original image geometry. A many-to-one hourglass network serves as the encode-decoder to fuse multiview features and generate vertex displacements as the fine-grained geometry. Besides, the neural network is trained by directly optimizing the visual effect, where two 3D shapes are compared by measuring the similarity between the multiview images rendered from the shapes. Finally, we propose to generate the ground-truth 3D shapes by registering RGB-D images followed by pose and shape augmentation, providing sufficient data for network training. Experiments on several challenging protocols demonstrate the superior reconstruction accuracy of our proposal on the face shape.

Application progress of artificial intelligence in traditional Chinese medicine nursing

“…On the one hand, existing works often rely on landmarks [17,59] and photometric-texture [12,44] to guide face reconstruction. In the case of extreme facial expressions, landmarks are sparse or inaccurate and the gradient from the texture loss cannot directly constrain the shape [58], posing a challenge for existing methods to achieve precise alignment of facial features in 3D face reconstruction, as depicted in Fig. 2(a).…”

Exploring semantic information for deep image aesthetics assessment

“…While for assessing facial aesthetics, we only focus on the texture and shape of the face itself [119], regardless of how the face is recorded in an image. And recent progress has been made to 3D face reconstruction [120][121][122], which makes assessing facial attractiveness in a finer-granularity to be possible. In the future, we will investigate how facial attractiveness can be assessed, especially from the perspective of 3D views.…”