Image-Guided Human Reconstruction via Multi-Scale Graph Transformation Networks

Abstract

3D human reconstruction from a single image is a challenging problem. Existing methods have difficulties to infer 3D clothed human models with consistent topologies for various poses. In this paper, we propose an efficient and effective method using a hierarchical graph transformation network. To deal with large deformations and avoid distorted geometries, rather than using Euclidean coordinates directly, 3D human shapes are represented by a vertex-based deformation representation that effectively encodes the deformation and copes well with large deformations. To infer a 3D human mesh consistent with the input real image, we also use a perspective projection layer to incorporate perceptual image features into the deformation representation. Our model is easy to train and fast to converge with short test time. Besides, we present the D 2 Human (Dynamic Detailed Human) dataset, including variously posed 3D human meshes with consistent topologies and rich geometry details, together with the captured color images and SMPL models, which is useful for training and evaluation of deep frameworks, particularly for graph neural networks. Experimental results demonstrate that our method achieves more plausible and complete 3D human reconstruction from a single image, compared with several state-of-the-art methods.

High-Fidelity Human Avatars from a Single RGB Camera

“…Alldieck et al [5] and Lazova et al [19] reconstructed a detailed parametric human model by solving an image-to-image translation problem to regress offsets in UV-space from a single RGB image, but they require a frontal photo as input and the recovered pose is restricted to A-pose. To obtain topology-consistent reconstruction for any pose, Li et al [21] propose a hierarchical graph transformation network. However, the reconstructed geometry and texture from a single RGB image is still smooth for unseen parts.…”

FOF: Learning Fourier Occupancy Field for Monocular Real-time Human Reconstruction

“…Their results are always in T-pose and cannot match the input images well. Li et al [14] reconstruct the geometry of a clothed human body with graph neural networks, but the recovered mesh tends to be smooth and lacks fine details. Zhu et al [32,33] use four stages constrained by joints, silhouettes and shading, but their results are sometimes inconsistent with the images.…”

“…Their results are robust but they are unable to reconstruct the clothes. Many classic representations, such as voxel grids [31] or meshes [33,2,14], have been exploited for monocular human reconstruction. However, the representation of voxel grids has to store discrete spatial grid samplings in memory.…”

3D Design Platform of Virtual National Costume Based on Digital Nonlinear Random Matrix

“…It is proposed to introduce the SDT into the tolerance mathematical modeling and systematically study the position degree of the shaft's hole feature. Different constraints of tolerance under MMR and LMR conditions and the corresponding position tolerance model are given [22]. Starting from the difficulty of solving the parameters in the SDT model, scholars proposed a method for solving the cylindricity tolerance model based on the combination of Monte Carlo and the response surface method [23].…”