360-Degree Textures of People in Clothing from a Single Image

Abstract

DensePose Garment segmentation Partial texture Completed texture Partial segmentation Completed segmentation Displacement maps Input view Fully-textured 3D avatar Figure 1: Given a single view of a person we predict a complete texture map in the UV space, complete clothing segmentation as well as a displacement map for the SMPL model [41], which we then combine to obtain a fully-textured 3D avatar. AbstractIn this paper we predict a full 3D avatar of a person from a single image. We infer texture and geometry in the UVspace of the SMPL model using an image-to-image translation method. Given partial texture and segmentation layout maps derived from the input view, our model predicts the complete segmentation map, the complete texture map, and a displacement map. The predicted maps can be applied to the SMPL model in order to naturally generalize to novel poses, shapes, and even new clothing. In order to learn our model in a common UV-space, we non-rigidly register the SMPL model to thousands of 3D scans, effectively encoding textures and geometries as images in correspondence. This turns a difficult 3D inference task into a simpler image-toimage translation one. Results on rendered scans of people and images from the DeepFashion dataset demonstrate that our method can reconstruct plausible 3D avatars from a single image. We further use our model to digitally change pose, shape, swap garments between people and edit clothing. To encourage research in this direction we will make the source code available for research purpose [5].

Detailed 3D human body reconstruction from multi-view images combining voxel super-resolution and learned implicit representation

“…There were some other approaches in which various cues were used for building sufficient loss function to train the network including the mesh [31], the texture [44], the multi-view images [34], the optimized SMPL model [30] and the video [27,29]. In order to model the detailed appearance, some method attempt to refine the regressed SMPL model to obtain the detailed 3D model [1,3,23,32,42,53,61,62]. In [1,3,32], after estimating the pose and shape of SMPL model, the authors used shape from shading and texture translation to add the details to SMPL like face, hairstyle, and clothes with garment wrinkles.…”

“…In order to model the detailed appearance, some method attempt to refine the regressed SMPL model to obtain the detailed 3D model [1,3,23,32,42,53,61,62]. In [1,3,32], after estimating the pose and shape of SMPL model, the authors used shape from shading and texture translation to add the details to SMPL like face, hairstyle, and clothes with garment wrinkles. In addition, the explicit representation of 3D human body model were also used in detailed reconstruction.…”

Neural3D: Light-weight Neural Portrait Scanning via Context-aware Correspondence Learning

“…However, the active IR-based depth sensors are unsuitable for outdoor capture, and their high power consumption limits the mobile application. Recently, with the advent of deep neural networks, purely RGB-based monocular methods have been proposed to encode various prior information of human models such as motion [23,24,50], geometry [20,28,36,37], garment [8] or appearance [25]. However, such methods still relies on per-vertex [1,[39][40][41] bring huge potential to enable more realistic 2D rendering results in the novel views.…”

FAKIR: An algorithm for revealing the anatomy and pose of statues from raw point sets

“…Similarly it is possible to register two models using manifold‐harmonics based non rigid registration [LRB*16], but this would not help for skeleton‐based registration. Learning approaches can also work from a single image [LIPM19, AMB *19].…”