A Dual-Source Approach for 3D Pose Estimation from a Single Image

Yasin, Hashim; Iqbal, Umar; Krüger, Björn; Weber, Andreas; Gall, Jüergen

doi:10.1109/cvpr.2016.535

Cited by 183 publications

(209 citation statements)

References 32 publications

Supporting

Mentioning

205

Contrasting

Order By: Relevance

“…The second category [1,29,4,15,10,6,25,17] decouples 3D pose estimation into the well-studied 2D joint detection [18,24] and 3D pose estimation from the detected 2D joints. Akhter et al [1] propose a multi-stage approach to estimate the 3D pose from 2D joints using an over-complete dictionary of poses.…”

Section: Related Workmentioning

confidence: 99%

“…Bogo et al [4] estimate 3D pose by first fitting a statistical body shape model to the 2D joints, and then minimizing the error between the reprojected 3D model and detected 2D joints. Chen [6] and Yasin [25] regard 3D pose estimation as a matching between the estimated 2D pose and the 3D pose from a large pose library. Martinez et al [15] design a simple fully connected residual network to regress 3D pose from 2D joint detections.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Generating Multiple Hypotheses for 3D Human Pose Estimation With Mixture Density Network

Lee

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

156

108

View full text Add to dashboard Cite

3D human pose estimation from a monocular image or 2D joints is an ill-posed problem because of depth ambiguity and occluded joints. We argue that 3D human pose estimation from a monocular input is an inverse problem where multiple feasible solutions can exist. In this paper, we propose a novel approach to generate multiple feasible hypotheses of the 3D pose from 2D joints. In contrast to existing deep learning approaches which minimize a mean square error based on an unimodal Gaussian distribution, our method is able to generate multiple feasible hypotheses of 3D pose based on a multimodal mixture density networks. Our experiments show that the 3D poses estimated by our approach from an input of 2D joints are consistent in 2D reprojections, which supports our argument that multiple solutions exist for the 2D-to-3D inverse problem. Furthermore, we show state-of-the-art performance on the Hu-man3.6M dataset in both best hypothesis and multi-view settings, and we demonstrate the generalization capacity of our model by testing on the MPII and MPI-INF-3DHP datasets. Our code is available at the project website 1 .

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Generating Multiple Hypotheses for 3D Human Pose Estimation With Mixture Density Network

Lee

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

156

108

View full text Add to dashboard Cite

show abstract

“…Sminchisescu further utilized temporal consistency to propagate pose probabilities with a Bayesian mixture of experts Markov model [2007]. Relying on the recent advances in machine learning techniques and compute capabilities, approaches for direct 3D pose regression from the input image have been proposed, using structured learning of latent pose [Li et al 2015a;Tekin et al 2016a], joint prediction of 2D and 3D pose [Li and Chan 2014;Tekin et al 2016b;Yasin et al 2016], transfer of features from 2D datasets [Mehta et al 2016], novel pose space formulations [Pavlakos et al 2016] and classification over example poses [Pons-Moll et al 2014;Rogez and Schmid 2016]. Relative per-bone predictions [Li and Chan 2014], kinematic skeleton models , or root centered joint positions [Ionescu et al 2014a] are used as the eventual output space.…”

Section: Multi-viewmentioning

confidence: 99%

“…Given 2D joint locations, lifting them to 3D pose is challenging. Existing approaches use bone length and depth ordering constraints [Mori and Malik 2006;Taylor 2000], sparsity assumptions [Wang et al 2014;Zhou et al 2015,a], joint limits [Akhter and Black 2015], inter-penetration constraints [Bogo et al 2016], temporal dependencies [Rhodin et al 2016b], and regression [Yasin et al 2016]. Treating 3D pose as a hidden variable in 2D estimation is an alternative [Brau and Jiang 2016].…”

Section: Multi-viewmentioning

confidence: 99%

VNect

et al. 2017

View full text Add to dashboard Cite

Fig. 1. We recover the full global 3D skeleton pose in real-time from a single RGB camera, even wireless capture is possible by streaming from a smartphone (left). It enables applications such as controlling a game character, embodied VR, sport motion analysis and reconstruction of community video (right). Community videos (CC BY) courtesy of Real Madrid C.F. [2016] and RUSFENCING-TV [2017].We present the first real-time method to capture the full global 3D skeletal pose of a human in a stable, temporally consistent manner using a single RGB camera. Our method combines a new convolutional neural network (CNN) based pose regressor with kinematic skeleton fitting. Our novel fullyconvolutional pose formulation regresses 2D and 3D joint positions jointly in real time and does not require tightly cropped input frames. A real-time kinematic skeleton fitting method uses the CNN output to yield temporally stable 3D global pose reconstructions on the basis of a coherent kinematic skeleton. This makes our approach the first monocular RGB method usable in real-time applications such as 3D character control-thus far, the only monocular methods for such applications employed specialized RGB-D cameras. Our method's accuracy is quantitatively on par with the best offline 3D monocular RGB pose estimation methods. Our results are qualitatively comparable to, and sometimes better than, results from monocular RGB-D approaches, such as the Kinect. However, we show that our approach is more broadly applicable than RGB-D solutions, i.e., it works for outdoor scenes, community videos, and low quality commodity RGB cameras.

show abstract

“…We follow the standard steps to align the 3D pose prediction with the groundtruth by aligning the position of the central hip joint, and use the Mean Per-Joint Position Error (MPJPE) between the groundtruth and the prediction as evaluation metrics. In some prior works [10], [41], [54], the pose prediction was further aligned with the groundtruth via a rigid transformation. The resulting MPJPE is termed as Procrustes Aligned (PA) MPJPE.…”

Section: Datasets and Evaluation Protocolsmentioning

confidence: 99%

HEMlets Pose: Learning Part-Centric Heatmap Triplets for Accurate 3D Human Pose Estimation

Zhou

Han

Jiang

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

Estimating 3D human pose from a single image is a challenging task. This work attempts to address the uncertainty of lifting the detected 2D joints to the 3D space by introducing an intermediate state -Part-Centric Heatmap Triplets (HEMlets), which shortens the gap between the 2D observation and the 3D interpretation. The HEMlets utilize three joint-heatmaps to represent the relative depth information of the end-joints for each skeletal body part. In our approach, a Convolutional Network (ConvNet) is first trained to predict HEMlets from the input image, followed by a volumetric joint-heatmap regression. We leverage on the integral operation to extract the joint locations from the volumetric heatmaps, guaranteeing end-to-end learning. Despite the simplicity of the network design, the quantitative comparisons show a significant performance improvement over the best-of-grade methods (e.g. 20% on Human3.6M). The proposed method naturally supports training with "in-the-wild" images, where only weakly-annotated relative depth information of skeletal joints is available. This further improves the generalization ability of our model, as validated by qualitative comparisons on outdoor images. Leveraging the strength of the HEMlets pose estimation, we further design and append a shallow yet effective network module to regress the SMPL parameters of the body pose and shape. We term the entire HEMlets-based human pose and shape recovery pipeline HEMlets PoSh. Extensive quantitative and qualitative experiments on the existing human body recovery benchmarks justify the state-of-the-art results obtained with our HEMlets PoSh approach.

show abstract

A Dual-Source Approach for 3D Pose Estimation from a Single Image

Cited by 183 publications

References 32 publications

Generating Multiple Hypotheses for 3D Human Pose Estimation With Mixture Density Network

Generating Multiple Hypotheses for 3D Human Pose Estimation With Mixture Density Network

VNect

HEMlets Pose: Learning Part-Centric Heatmap Triplets for Accurate 3D Human Pose Estimation

Contact Info

Product

Resources

About