Learning from Synthetic Humans

Abstract

Estimating human pose, shape, and motion from images and videos are fundamental challenges with many applications. Recent advances in 2D human pose estimation use large amounts of manually-labeled training data for learning convolutional neural networks (CNNs). Such data is time consuming to acquire and difficult to extend. Moreover, manual labeling of 3D pose, depth and motion is impractical. In this work we present SURREAL (Synthetic hUmans foR REAL tasks): a new large-scale dataset with synthetically-generated but realistic images of people rendered from 3D sequences of human motion capture data. We generate more than 6 million frames together with ground truth pose, depth maps, and segmentation masks. We show that CNNs trained on our synthetic dataset allow for accurate human depth estimation and human part segmentation in real RGB images. Our results and the new dataset open up new possibilities for advancing person analysis using cheap and large-scale synthetic data.

Learning‐Based Animation of Clothing for Virtual Try‐On

“…Zurdo et al [ZBO13] proposed a mapping between low and high-resolution simulations, employing tracking constraints [BMWG07] to establish a correspondence between both resolutions. Saito et al [SUM14] proposed an upsampling technique that adds physically feasible microscopic detail to coarsened meshes by considering the internal strain at runtime. More recently, Oh et al [OLL18] have shown how to train a deep neural network to upsample low-resolution cloth simulations.…”

Robust Human Pose Tracking For Realistic Service Robot Applications

“…They achieve a large computation gain by substituting pixel-wise classification with the estimation of the probability distribution to the direction towards a particular joint. Meanwhile, in [24,37] large synthetic depth human motion datasets are introduced, leading to CNN-based body part estimators from depth data.…”

“…Future improvements of the proposed framework may include incorporation of the discriminative estimator within the optimization process and enhancement of the accuracy utilizing CNNs [24,37], which have achieved impressive results in identification tasks [20]. Further elaboration of the data preprocessing algorithms can also increase robustness, while from a h/w standpoint, performance modelling on various architectures may potentially provide significant benefits, as achieving robust and accurate pose estimation on lower-spec h/w can be of major significance in the case of applications where the available processing power is limited, such as autonomous robots.…”

HEMlets Pose: Learning Part-Centric Heatmap Triplets for Accurate 3D Human Pose Estimation

“…First, some works directly tackled the issue by putting efforts on constructing target datasets. The work of [47] firstly built a synthetic dataset using graphics techniques, however, training only on this dataset is still difficult to produce a model that is applicable to real images. Lassner et al [20] proposed to apply the algorithm of [4] to obtain 3D body models for real images and then manually sift out the reasonable results, to build the final human body dataset.…”

“…In this part, we evaluate the proposed human body recovery method of regressing the SMPL parameters on three public datasets i.e., SURREAL [47], UP-3D [20] and 3DPW [48]. Before the experimental studies, we first give an introduction to the datasets and related evaluation protocols.…”