Integral Human Pose Regression

Sun, Xiao; Xiao, Bin; Wei, Fangyin; Liang, Shuang; Wei, Yichen

doi:10.1007/978-3-030-01231-1_33

Cited by 666 publications

(658 citation statements)

References 66 publications

Supporting

Mentioning

623

Contrasting

Order By: Relevance

“…With a simple network architecture and little parameter tuning, we produce the most competitive results compared to previous works with carefully designed networks powered by e.g., adversarial training schemes or prior knowledge. On average, we improve the 3D pose prediction accuracy by 20% than that reported in Sun et al [41] under Protocol #1. We also report our performance using PA MPJPE as the evaluation [14] under Protocol #1 and Protocol #2, as well as using PA MPJPE as the evaluation metric.…”

Section: Results and Comparisonsmentioning

confidence: 43%

“…It is proved very helpful to bridge the input 2D image and the output 3D pose in the learning procedure. We demonstrated the effectiveness of the proposed method tested over the standard benchmarks, yielding a relative accuracy improvement of about 20% over one best-of-grade method [41] on the Human3.6M benchmark. Good generalization ability is also witnessed for the presented approach.…”

Section: Resultsmentioning

confidence: 91%

“…It usually suffers from quantization errors for the joint locations. To address this issue, Sun et al [41] exploited a soft-argmax operation and proposed an end-to-end training scheme for the 3D volumetric regression, achieving by far the best performance on 3D pose estimation. Inspired by [33] that the relative depth ordering across joints is helpful for resolving pose ambiguities, Pavlakos et al [30] adopted a ranking loss for pairwise ordinal depth to train the 3D human pose predictor explicitly.…”

Section: Body Pose Estimationmentioning

confidence: 99%

“…A straightforward strategy is to use volumetric heatmaps to represent the likelihood map of each 3D joint location [31]. Sun et al [41] further proposed a differentiable soft-argmax operator that unifies the joint detection task and the regression task into an end-to-end training framework. This significantly improves the state-of-the-art 3D pose estimation accuracy.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

HEMlets Pose: Learning Part-Centric Heatmap Triplets for Accurate 3D Human Pose Estimation

Zhou

Han

Jiang

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

101

View full text Add to dashboard Cite

Estimating 3D human pose from a single image is a challenging task. This work attempts to address the uncertainty of lifting the detected 2D joints to the 3D space by introducing an intermediate state -Part-Centric Heatmap Triplets (HEMlets), which shortens the gap between the 2D observation and the 3D interpretation. The HEMlets utilize three joint-heatmaps to represent the relative depth information of the end-joints for each skeletal body part. In our approach, a Convolutional Network (ConvNet) is first trained to predict HEMlets from the input image, followed by a volumetric joint-heatmap regression. We leverage on the integral operation to extract the joint locations from the volumetric heatmaps, guaranteeing end-to-end learning. Despite the simplicity of the network design, the quantitative comparisons show a significant performance improvement over the best-of-grade methods (e.g. 20% on Human3.6M). The proposed method naturally supports training with "in-the-wild" images, where only weakly-annotated relative depth information of skeletal joints is available. This further improves the generalization ability of our model, as validated by qualitative comparisons on outdoor images. Leveraging the strength of the HEMlets pose estimation, we further design and append a shallow yet effective network module to regress the SMPL parameters of the body pose and shape. We term the entire HEMlets-based human pose and shape recovery pipeline HEMlets PoSh. Extensive quantitative and qualitative experiments on the existing human body recovery benchmarks justify the state-of-the-art results obtained with our HEMlets PoSh approach.

show abstract

Section: Results and Comparisonsmentioning

confidence: 43%

Section: Resultsmentioning

confidence: 91%

Section: Body Pose Estimationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

HEMlets Pose: Learning Part-Centric Heatmap Triplets for Accurate 3D Human Pose Estimation

Zhou

Han

Jiang

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

101

View full text Add to dashboard Cite

show abstract

“…Pavlakos et al [42] proposed the volumetric stacked hourglass architecture, but the method suffers from significant increase in the number of parameters and from the required memory to store all the gradients. A similar technique is used in [55], but instead of using argmax for coordinate estimation, the authors use a numerical integral regression, which is similar to the soft-argmax operation [34]. More recently, Yang et al [65] proposed to use adversarial networks to distinguish between generated and ground truth poses, improving predictions on uncontrolled environments.…”

Section: Pose Estimationmentioning

confidence: 99%

Multi-task Deep Learning for Real-Time 3D Human Pose Estimation and Action Recognition

Luvizon

Picard

Tabia

2020

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Human pose estimation and action recognition are related tasks since both problems are strongly dependent on the human body representation and analysis. Nonetheless, most recent methods in the literature handle the two problems separately. In this work, we propose a multi-task framework for jointly estimating 2D or 3D human poses from monocular color images and classifying human actions from video sequences. We show that a single architecture can be used to solve both problems in an efficient way and still achieves state-of-the-art or comparable results at each task while running with a throughput of more than 100 frames per second. The proposed method benefits from high parameters sharing between the two tasks by unifying still images and video clips processing in a single pipeline, allowing the model to be trained with data from different categories simultaneously and in a seamlessly way. Additionally, we provide important insights for end-to-end training the proposed multi-task model by decoupling key prediction parts, which consistently leads to better accuracy on both tasks. The reported results on four datasets (MPII, Human3.6M, Penn Action and NTU RGB+D) demonstrate the effectiveness of our method on the targeted tasks. Our source code and trained weights are publicly available at

show abstract

SADNet: Generating immersive virtual reality avatars by real‐time monocular pose estimation

Jiang,

Xiong,

Wang

et al. 2024

Computer Animation & Virtual

View full text Add to dashboard Cite

SummaryGenerating immersive virtual reality avatars is a challenging task in VR/AR applications, which maps physical human body poses to avatars in virtual scenes for an immersive user experience. However, most existing work is time‐consuming and limited by datasets, which does not satisfy immersive and real‐time requirements of VR systems. In this paper, we aim to generate 3D real‐time virtual reality avatars based on a monocular camera to solve these problems. Specifically, we first design a self‐attention distillation network (SADNet) for effective human pose estimation, which is guided by a pre‐trained teacher. Secondly, we propose a lightweight pose mapping method for human avatars that utilizes the camera model to map 2D poses to 3D avatar keypoints, generating real‐time human avatars with pose consistency. Finally, we integrate our framework into a VR system, displaying generated 3D pose‐driven avatars on Helmet‐Mounted Display devices for an immersive user experience. We evaluate SADNet on two publicly available datasets. Experimental results show that SADNet achieves a state‐of‐the‐art trade‐off between speed and accuracy. In addition, we conducted a user experience study on the performance and immersion of virtual reality avatars. Results show that pose‐driven 3D human avatars generated by our method are smooth and attractive.

show abstract

Integral Human Pose Regression

Cited by 666 publications

References 66 publications

HEMlets Pose: Learning Part-Centric Heatmap Triplets for Accurate 3D Human Pose Estimation

HEMlets Pose: Learning Part-Centric Heatmap Triplets for Accurate 3D Human Pose Estimation

Multi-task Deep Learning for Real-Time 3D Human Pose Estimation and Action Recognition

SADNet: Generating immersive virtual reality avatars by real‐time monocular pose estimation

Contact Info

Product

Resources

About