Three-dimensional human shape reconstruction is important in many applications, such as virtual or augmented reality (VR/AR), virtual clothing fitting, and healthcare. In this paper, we propose a learning-based method for reconstructing a whole-body point cloud from a single front-view humandepth image. Because actual depth images typically suffer from noise and missing data, an accurate point cloud cannot be reasonably obtained by simply predicting a back-view depth image. To solve this problem, we propose to use convolutional neural networks that not only predict a back-view depth image but also refine the input front-view depth image. To train the networks, we propose a carefully designed method for generating synthetic but realistic human-depth images with noise and missing data. Experiments show that the proposed method is effective for obtaining seamless whole-body point clouds. In addition, the experiments show that the networks trained on the synthetic depth images are ready for application to actual depth images. INDEX TERMS 3D human shape, convolutional neural networks, deep learning, single depth image, synthetic data generation.