Weakly Supervised 3D Human Pose and Shape Reconstruction with Normalizing Flows

Zanfir, Andrei; Bazavan, Eduard Gabriel; Xu, Hongyi; Freeman, Bill; Sukthankar, Rahul; Sminchisescu, Cristian

doi:10.1007/978-3-030-58539-6_28

Cited by 100 publications

(81 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Segmentation in pose estimation. Segmentation has been used in 3D hand pose estimation, 3D human pose estimation and hand tracking and can be grouped into four categories: as a localization step [2,21,37,38,59,61], as a training loss [3,5], as an optimization term [7,35,56], or as an intermediate representation [8,36,41,44,58]. Most single hand pose estimation approaches follow Zimmermann et al [61] in localizing a hand in an image by predicting the hand silhouette, which is used to crop the input image before performing pose estimation.…”

Section: Related Workmentioning

confidence: 99%

“…Our method neither assumes RGB nor depth image sequences. In 3D human pose and shape estimation, existing methods predict part segmentation maps [41,58] or silhouettes [44] from RGB images and use the predicted masks as an intermediate representation.…”

Section: Related Workmentioning

confidence: 99%

“…We show that this reduces the ambiguities brought on by the selfsimilarity of hands and, in turn, significantly improves the accuracy of 3D pose estimates. While prior work on hand- [5,61] and body-pose estimation [41,44,58] and handtracking [14,35,56] have leveraged some form of segmentation, most often silhouettes or per-pixel masks, this is typically done as a pre-processing step. In contrast, our ablations show that integrating a semantic segmentation branch into an end-to-end trained architecture already increases pose estimation accuracy.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Learning To Disambiguate Strongly Interacting Hands via Probabilistic Per-Pixel Part Segmentation

Fan

Spurr

Kocabas

et al. 2021

2021 International Conference on 3D Vision (3DV)

View full text Add to dashboard Cite

https://zc-alexfan.github.io/digit Figure 1. When estimating the 3D pose of interacting hands, state-of-the-art methods struggle to disambiguate the appearance of the two hands and their parts. In this example, the baseline fails to differentiate the left and the right wrists (1.1), resulting in erroneous pose estimation (1.2). Our model, DIGIT, reduces the ambiguity by predicting and leveraging a probabilistic part segmentation volume (2.1) to produce reliable pose estimates even when the two hands are in direct contact and under significant occlusion (2.2, 2.3).

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Learning To Disambiguate Strongly Interacting Hands via Probabilistic Per-Pixel Part Segmentation

Fan

Spurr

Kocabas

et al. 2021

2021 International Conference on 3D Vision (3DV)

View full text Add to dashboard Cite

show abstract

“…Since video-based methods e.g. [22,54] assume massive inthe-wild human motions in SMPL format are available which may not be realistic in the real world, thus we stick to the image-based methods. Technically, the plug-and-play mesh2mesh is proposed to improve the temporal consistency of the mesh sequence without additional meshes as training data.…”

Section: Related Workmentioning

confidence: 99%

Can Action be Imitated? Learn to Reconstruct and Transfer Human Dynamics from Videos

Jiang

2021

Proceedings of the 2021 International Conference on Multimedia Retrieval

View full text Add to dashboard Cite

show abstract

“…Moreover, for image denoising and restoration, Zanfir et.al. [50] designed different normalizing flows-based prior representations, which were used for the first time in modeling a 3D human pose. Recently, C-flow [34] and Pointflow [46] integrated normalizing flows into 3D point clouds with considerable possibilities for multimodal data modeling.…”

Section: Normalizing Flowsmentioning

confidence: 99%

Learning Hierarchical Embedding for Video Instance Segmentation

Qin

Nie

et al. 2021

Proceedings of the 29th ACM International Conference on Multimedia

View full text Add to dashboard Cite

In this paper, we address video instance segmentation using a new generative model that learns effective representations of the target and background appearance. We propose to exploit hierarchical structural embedding over spatio-temporal space, which is compact, powerful, and flexible in contrast to current tracking-by-detection methods. Specifically, our model segments and tracks instances across space and time in a single forward pass, which is formulated as hierarchical embedding learning. The model is trained to locate the pixels belonging to specific instances over a video clip. We firstly take advantage of a novel mixing function to better fuse spatiotemporal embeddings. Moreover, we introduce normalizing flows to further improve the robustness of the learned appearance embedding, which theoretically extends conventional generative flows to a factorized conditional scheme. Comprehensive experiments on the video instance segmentation benchmark, i.e., YouTube-VIS, demonstrate the effectiveness of the proposed approach. Furthermore, we evaluate our method on an unsupervised video object segmentation dataset to demonstrate its generalizability.

show abstract

Weakly Supervised 3D Human Pose and Shape Reconstruction with Normalizing Flows

Cited by 100 publications

References 37 publications

Learning To Disambiguate Strongly Interacting Hands via Probabilistic Per-Pixel Part Segmentation

Learning To Disambiguate Strongly Interacting Hands via Probabilistic Per-Pixel Part Segmentation

Can Action be Imitated? Learn to Reconstruct and Transfer Human Dynamics from Videos

Learning Hierarchical Embedding for Video Instance Segmentation

Contact Info

Product

Resources

About