Self-Supervised 3D Human Pose Estimation via Part Guided Novel Image Synthesis

Kundu, Jogendra Nath; Seth, Siddharth; Jampani, Varun; Rakesh, Mugalodi; Babu, R. Venkatesh; Chakraborty, Anirban

doi:10.1109/cvpr42600.2020.00619

Cited by 66 publications

(19 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Pavlakos et al [48] Kundu et al [39] Bogo et al [34] Rhodin et al [43] Zhou et al [57] Wandt et al [58] Chen et al [59] Iskakov et al [52] Kocabas et al [40] Kundu et al [41] Rhodin et al [42] Guler et al [60] Pavlakos et al [19] Weakly-supervised…”

Section: Iqbal Et Al [62]mentioning

confidence: 99%

“…EpipolarPose simultaneously takes two consecutive images while training, but it takes a single image during inference as a monocular method. In Reference [41], a differentiable and modular self-supervised learning paradigm is proposed. The whole network contains an encoder and a decoder.…”

Section: Unsupervisedmentioning

confidence: 99%

“…To be noted is that, in some works, other supervision forms can change to semi-supervised ways. For example, Reference [41] utilizes 10% of the full training set with direct 3D pose supervision referring to the semi-supervised setting.…”

Section: Semi-supervisedmentioning

confidence: 99%

See 2 more Smart Citations

Deep Learning Methods for 3D Human Pose Estimation under Different Supervision Paradigms: A Survey

Zhang

Guo

et al. 2021

Electronics

View full text Add to dashboard Cite

The rise of deep learning technology has broadly promoted the practical application of artificial intelligence in production and daily life. In computer vision, many human-centered applications, such as video surveillance, human-computer interaction, digital entertainment, etc., rely heavily on accurate and efficient human pose estimation techniques. Inspired by the remarkable achievements in learning-based 2D human pose estimation, numerous research studies are devoted to the topic of 3D human pose estimation via deep learning methods. Against this backdrop, this paper provides an extensive literature survey of recent literature about deep learning methods for 3D human pose estimation to display the development process of these research studies, track the latest research trends, and analyze the characteristics of devised types of methods. The literature is reviewed, along with the general pipeline of 3D human pose estimation, which consists of human body modeling, learning-based pose estimation, and regularization for refinement. Different from existing reviews of the same topic, this paper focus on deep learning-based methods. The learning-based pose estimation is discussed from two categories: single-person and multi-person. Each one is further categorized by data type to the image-based methods and the video-based methods. Moreover, due to the significance of data for learning-based methods, this paper surveys the 3D human pose estimation methods according to the taxonomy of supervision form. At last, this paper also enlists the current and widely used datasets and compares performances of reviewed methods. Based on this literature survey, it can be concluded that each branch of 3D human pose estimation starts with fully-supervised methods, and there is still much room for multi-person pose estimation based on other supervision methods from both image and video. Besides the significant development of 3D human pose estimation via deep learning, the inherent ambiguity and occlusion problems remain challenging issues that need to be better addressed.

show abstract

Section: Iqbal Et Al [62]mentioning

confidence: 99%

Section: Unsupervisedmentioning

confidence: 99%

See 1 more Smart Citation

Deep Learning Methods for 3D Human Pose Estimation under Different Supervision Paradigms: A Survey

Zhang

Guo

et al. 2021

Electronics

View full text Add to dashboard Cite

show abstract

“…Kocabas et al, 2019;Mitra et al, 2020;Rhodin et al, 2018) or the time-consistency (Pavllo et al, 2019). Kundu et al (2020) developed a self-supervised method via synthesize novel images to reduce the data bias and improve the generalization. Li et al (2020) proposed a weakly-supervised method by generating novel 2D-3D human poses pairs.…”

Section: Weakly or Semi-supervised 3d Human Pose Estimationmentioning

confidence: 99%

Lifting 2D Human Pose to 3D with Domain Adapted 3D Body Concept

Nie

Liu

2021

Preprint

View full text Add to dashboard Cite

Lifting the 2D human pose to the 3D pose is an important yet challenging task. Existing 3D pose estimation suffers from 1) the inherent ambiguity between the 2D and 3D data, and 2) the lack of well labeled 2D-3D pose pairs in the wild. Human beings are able to imagine the human 3D pose from a 2D image or a set of 2D body key-points with the least ambiguity, which should be attributed to the prior knowledge of the human body that we have acquired in our mind. Inspired by this, we propose a new framework that leverages the labeled 3D human poses to learn a 3D concept of the human body to reduce the ambiguity. To have consensus on the body concept from 2D pose, our key insight is to treat the 2D human pose and the 3D human pose as two different domains. By adapting the two domains, the body knowledge learned from 3D poses is applied to 2D poses and guides the 2D pose encoder to generate informative 3D "imagination" as embedding in pose lifting. Benefiting from the domain adaptation perspective, the proposed framework unifies the supervised and semi-supervised 3D pose estimation in a principled framework. Extensive experiments demonstrate that the proposed approach can achieve state-of-the-art performance on standard benchmarks. More importantly, it is validated that the explicitly learned 3D body concept effectively alleviates the 2D-3D ambiguity in 2D pose lifting, improves the generalization, and enables the network to exploit the abundant unlabeled 2D data.

show abstract

“…Weakly and self-supervised learning methods relaxed the need of 3D ground-truth body poses by exploiting unpaired 2D and 3D body poses [28,8,26] or multi-view images [5,15,17]. Nevertheless, not a single method explores the E-mail: firstname.lastname@{daimler.com, uni-ulm.de}.…”

Section: Introductionmentioning

confidence: 99%

Learning Temporal 3D Human Pose Estimation with Pseudo-Labels

Bouazizi¹,

Kreßel²,

Belagiannis³

2021

Preprint

View full text Add to dashboard Cite

We present a simple, yet effective, approach for selfsupervised 3D human pose estimation. Unlike the prior work, we explore the temporal information next to the multi-view self-supervision. During training, we rely on triangulating 2D body pose estimates of a multiple-view camera system. A temporal convolutional neural network is trained with the generated 3D ground-truth and the geometric multi-view consistency loss, imposing geometrical constraints on the predicted 3D body skeleton. During inference, our model receives a sequence of 2D body pose estimates from a singleview to predict the 3D body pose for each of them. An extensive evaluation shows that our method achieves state-of-theart performance in the Human3.6M and MPI-INF-3DHP benchmarks. Our code and models are publicly available at https://github.com/vru2020/TM_HPE/.

show abstract

Self-Supervised 3D Human Pose Estimation via Part Guided Novel Image Synthesis

Cited by 66 publications

References 55 publications

Deep Learning Methods for 3D Human Pose Estimation under Different Supervision Paradigms: A Survey

Deep Learning Methods for 3D Human Pose Estimation under Different Supervision Paradigms: A Survey

Lifting 2D Human Pose to 3D with Domain Adapted 3D Body Concept

Learning Temporal 3D Human Pose Estimation with Pseudo-Labels

Contact Info

Product

Resources

About