No abstract

While there has been a success in 2D human pose estimation with convolutional neural networks (CNNs), 3D human pose estimation has not been thoroughly studied. In this paper, we tackle the 3D human pose estimation task with end-to-end learning using CNNs. Relative 3D positions between one joint and the other joints are learned via CNNs. The proposed method improves the performance of CNN with two novel ideas. First, we added 2D pose information to estimate a 3D pose from an image by concatenating 2D pose estimation result with the features from an image. Second, we have found that more accurate 3D poses are obtained by combining information on relative positions with respect to multiple joints, instead of just one root joint. Experimental results show that the proposed method achieves comparable performance to the state-of-the-art methods on Human 3.6m dataset.

Non-rigid structure from motion (NRSfM) is a fundamental problem of computer vision. Recently, it has been shown that incorporating shape alignment in NRSfM can improve the performance significantly compared with the other algorithms, which do not consider shape alignment. However, realizing this idea was at a cost of a heavy, complicated process, which limits its usefulness and possible extensions. In this paper, we propose a novel regression framework for NRSfM, of which the variables (3D shapes) are regularized based on their aligned shapes. We show that this can be casted into an unconstrained problem or a problem with simple bound constraints, which can be efficiently solved by existing solvers. This framework can be easily integrated with numerous existing models and assumptions, such as orthographic or perspective camera models, occlusion, low-rank assumption, smooth deformations, and so on, which makes it more practical for various real situations. The experimental results show that the proposed method gives competitive result to the state-of-the-art methods for orthographic projection with much less time complexity and memory requirement, and outperforms the existing methods for perspective projection.

For human pose estimation in videos, it is significant how to use temporal information between frames. In this paper, we propose temporal flow maps for limbs (TML) and a multi-stride method to estimate and track human poses. The proposed temporal flow maps are unit vectors describing the limbs' movements. We constructed a network to learn both spatial information and temporal information end-to-end. Spatial information such as joint heatmaps and part affinity fields is regressed in the spatial network part, and the TML is regressed in the temporal network part. We also propose a data augmentation method to learn various types of TML better. The proposed multi-stride method expands the data by randomly selecting two frames within a defined range. We demonstrate that the proposed method efficiently estimates and tracks human poses on the PoseTrack 2017 and 2018 datasets.

No abstract