A Neural Network for Detailed Human Depth Estimation From a Single Image

Abstract

This paper presents a neural network to estimate a detailed depth map of the foreground human in a single RGB image. The result captures geometry details such as cloth wrinkles, which are important in visualization applications. To achieve this goal, we separate the depth map into a smooth base shape and a residual detail shape and design a network with two branches to regress them respectively. We design a training strategy to ensure both base and detail shapes can be faithfully learned by the corresponding network branches. Furthermore, we introduce a novel network layer to fuse a rough depth map and surface normals to further improve the final result. Quantitative comparison with fused 'ground truth' captured by real depth cameras and qualitative examples on unconstrained Internet images demonstrate the strength of the proposed method. Our code will be released at Link

Unsupervised Single-shot Depth Estimation using Perceptual Reconstruction

“…Leveraging the task to whole body depth estimation is challenging due to the fact that RGB-depth pairs of real individuals are not abundant in many datasets. A small dataset of 25 video clips for detailed human depth estimation is proposed in [10] while a depth dataset of 10 sequences recorded from different viewpoints is published in [8]. The Human3.6M dataset [13] contains high-resolution depth data from 11 individuals acting in varying scenarios.…”

“…Real-time depth inference of a given object is a highly important computer vision task which can be applied in various robotic tasks such as simultaneous localization and mapping [1,2,3] as well as autonomous quality inspection in industrial applications [4,5]. As the popularity of VR applications has continued to grow, instant depth estimation has also become an integral part of modeling complex 3D information out of single 2D images of human faces [6,7] or body parts [8,9,10]. Depth information about an object can be directly obtained from sensors for optical distance measurement.…”

“…Depth information about an object can be directly obtained from sensors for optical distance measurement. Time-of-Flight (ToF) cameras, LIDAR or stereo imaging systems are often used in practice and were also employed to generate paired RGB-depth data from some well-known depth databases [1,2,8,10,11,12,13]. Since these sensors are typically very expensive time-consuming devices that are also subject to external influences, their applicability to fast full-image depth generation on small on-site devices is limited.…”

“…Although DCNNs are a promising approach that succeed on such transfer tasks, they are commonly based on large amounts of training data, and generation and acquisition can be a demanding task. In the supervised setting in particular, DCNNs make use of paired training data during network parameter optimization, i.e., the network is provided with a single-view RGB and corresponding per-pixel depth [6,10,14,15]. Since large scale dense depth profiles are not abundant in many applications, supervised approaches are not feasible for these objects.…”

Deep Fashion3D: A Dataset and Benchmark for 3D Garment Reconstruction from Single Images

“…Attempts have been made to jointly reconstruct body and clothing from videos [3,4] and multi-view images [29,62]. Recent advances in deep learning based approaches [44,56,51,5,35,2,50,14,55] have achieved single-view clothed body reconstruction. However, for all these methods, tedious manual post-processing is required to extract the clothing surface from the reconstructed result.…”