Appearance Consensus Driven Self-supervised Human Mesh Recovery

Kundu, Jogendra Nath; Rakesh, Mugalodi; Jampani, Varun; Venkatesh, R.; Babu, R. Venkatesh

doi:10.1007/978-3-030-58452-8_46

Cited by 30 publications

(16 citation statements)

References 62 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our method outperformed previous methods in all three types of supervision settings (i.e., self/weakly/semi-supervised). Kundu et al [64]'s method is the competitive self-supervised learning framework using the video priors; however, our method outperformed it even without any video priors. Li et al [54]'s method is the only baseline that outperforms our method and uses the self-supervised setting.…”

Section: Resultsmentioning

confidence: 63%

“…For the Human3.6M dataset, we compare our method to recent fully/semi/weakly-/self-supervised methods [5,36,37,43,[47][48][49][50]64] that output the 3D human poses by protocol-2 in Table 6. Our method outperformed previous methods in all three types of supervision settings (i.e., self/weakly/semi-supervised).…”

Section: Resultsmentioning

confidence: 99%

“…To show the generalization ability of our method, we evaluated it on 3DPW, the data of which were never seen in our training process. For the 3DPW dataset, we compared the accuracy of our method to those of recent state-of-the-art approaches [4,5,9,47,51,64] in Table 7. For the LSP dataset, as the dataset does not contain 3D pose annotations, we only measured the accuracy and F1 score of FG-BG segmentation and six-part segmentation, respectively, for comparison with other algorithms.…”

Section: Resultsmentioning

confidence: 99%

“…For the LSP dataset, as the dataset does not contain 3D pose annotations, we only measured the accuracy and F1 score of FG-BG segmentation and six-part segmentation, respectively, for comparison with other algorithms. We compared our method to state-ofthe-art optimized-based methods [9,37,65] and regression-based methods [5,49,50,64] in Table 8. Optimization-based methods tend to have better performance on segmentation tasks than regression-based methods; however, our method outperformed most of them even in the self-supervised setting (i.e., Ours (self-sup)); our approach was also recorded as the best when involving more supervisions (i.e., Ours (weakly-sup) and Ours (semi-sup)).…”

Section: Resultsmentioning

confidence: 99%

See 3 more Smart Citations

Towards Single 2D Image-Level Self-Supervision for 3D Human Pose and Shape Estimation

et al. 2021

View full text Add to dashboard Cite

Three-dimensional human pose and shape estimation is an important problem in the computer vision community, with numerous applications such as augmented reality, virtual reality, human computer interaction, and so on. However, training accurate 3D human pose and shape estimators based on deep learning approaches requires a large number of images and corresponding 3D ground-truth pose pairs, which are costly to collect. To relieve this constraint, various types of weakly or self-supervised pose estimation approaches have been proposed. Nevertheless, these methods still involve supervision signals, which require effort to collect, such as unpaired large-scale 3D ground truth data, a small subset of 3D labeled data, video priors, and so on. Often, they require installing equipment such as a calibrated multi-camera system to acquire strong multi-view priors. In this paper, we propose a self-supervised learning framework for 3D human pose and shape estimation that does not require other forms of supervision signals while using only single 2D images. Our framework inputs single 2D images, estimates human 3D meshes in the intermediate layers, and is trained to solve four types of self-supervision tasks (i.e., three image manipulation tasks and one neural rendering task) whose ground-truths are all based on the single 2D images themselves. Through experiments, we demonstrate the effectiveness of our approach on 3D human pose benchmark datasets (i.e., Human3.6M, 3DPW, and LSP), where we present the new state-of-the-art among weakly/self-supervised methods.

show abstract

Section: Resultsmentioning

confidence: 63%

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

See 2 more Smart Citations

Towards Single 2D Image-Level Self-Supervision for 3D Human Pose and Shape Estimation

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Zhu et al [172] combined the SMPL model with a hierarchical mesh deformation framework to enhance the flexibility of free-form 3D deformation. Kundu et al [173] included a colorrecovery module in the SMPL model to obtain vertex color via reflectional symmetry. Arnab et al [113] pointed out that methods using the SMPL model usually fail on the in-thewild data.…”

Section: A Single-person 3d Hpementioning

confidence: 99%

Deep Learning-Based Human Pose Estimation: A Survey

Zheng¹,

Wu²,

Yang³

et al. 2020

Preprint

View full text Add to dashboard Cite

Human pose estimation aims to locate the human body parts and build human body representation (e.g., body skeleton) from input data such as images and videos. It has drawn increasing attention during the past decade and has been utilized in a wide range of applications including human-computer interaction, motion analysis, augmented reality, and virtual reality. Although the recently developed deep learning-based solutions have achieved high performance in human pose estimation, there still remain challenges due to insufficient training data, depth ambiguities, and occlusion. The goal of this survey paper is to provide a comprehensive review of recent deep learning-based solutions for both 2D and 3D pose estimation via a systematic analysis and comparison of these solutions based on their input data and inference procedures. More than 240 research papers since 2014 are covered in this survey. Furthermore, 2D and 3D human pose estimation datasets and evaluation metrics are included. Quantitative performance comparisons of the reviewed methods on popular datasets are summarized and discussed. Finally, the challenges involved, applications, and future research directions are concluded. We also provide a regularly updated project page: https://github.com/zczcwh/DL-HPE

show abstract