Monocular 3D Pose and Shape Estimation of Multiple People in Natural Scenes: The Importance of Multiple Scene Constraints

Zanfir, Andrei; Marinoiu, Elisabeta; Sminchisescu, Cristian

doi:10.1109/cvpr.2018.00229

Cited by 295 publications

(205 citation statements)

References 29 publications

Supporting

Mentioning

198

Contrasting

Order By: Relevance

“…In [20], a single image and corresponding landmarks are used to lookup a similar human pose using a kd-tree, containing about 4 million examples. A method intended for multi-instance model fitting from a single image is described in [21].…”

Section: Closely Related Workmentioning

confidence: 99%

3D Human Body Reconstruction from a Single Image via Volumetric Regression

Jackson

Manafas²,

Tzimiropoulos

2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

This paper proposes the use of an end-to-end Convolutional Neural Network for direct reconstruction of the 3D geometry of humans via volumetric regression. The proposed method does not require the fitting of a shape model and can be trained to work from a variety of input types, whether it be landmarks, images or segmentation masks. Additionally, non-visible parts, either self-occluded or otherwise, are still reconstructed, which is not the case with depth map regression. We present results that show that our method can handle both pose variation and detailed reconstruction given appropriate datasets for training. Fig. 1. Some example results using our method when trained with a high quality detailed training set.

show abstract

Section: Closely Related Workmentioning

confidence: 99%

3D Human Body Reconstruction from a Single Image via Volumetric Regression

Jackson

Manafas²,

Tzimiropoulos

2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…Our method, on the other hand, avoids the use of such datasets and relies on 3D data only for the single person case. Further, Zanfir et al [37] proposed a largescale human sensing system for multiple people that estimates pose and shape using the top-down approach of person detection followed by pose estimation for each person. Recently, Zanfir et al [38] proposed MubyNet, a bottom-up approach that performs joint association by formulating it as a binary integer programming problem.…”

Section: Related Workmentioning

confidence: 99%

“…Unfortunately, the run-time of this approach is likely to increase linearly with the number of people in the scene, making it inefficient for analysis in crowded scenes. Additionally, most existing multi-person pose estimation methods [27,20,28], with the exception of [37] estimate 3D pose configuration only relative to the root joint. However, relative spatial ordering of different people in the scene is also needed to facilitate reasoning about human interactions and provide a better understanding of the scene.…”

Section: Introductionmentioning

confidence: 99%

Multi-Person 3D Human Pose Estimation from Monocular Images

Dabral

Gundavarapu

Mitra

et al. 2019

2019 International Conference on 3D Vision (3DV)

View full text Add to dashboard Cite

Multi-person 3D human pose estimation from a single image is a challenging problem, especially for in-the-wild settings due to the lack of 3D annotated data. We propose HG-RCNN, a Mask-RCNN based network that also leverages the benefits of the Hourglass architecture for multiperson 3D Human Pose Estimation. A two-staged approach is presented that first estimates the 2D keypoints in every Region of Interest (RoI) and then lifts the estimated keypoints to 3D. Finally, the estimated 3D poses are placed in camera-coordinates using weak-perspective projection assumption and joint optimization of focal length and root translations. The result is a simple and modular network for multi-person 3D human pose estimation that does not require any multi-person 3D pose dataset. Despite its simple formulation, HG-RCNN achieves the state-of-the-art results on MuPoTS-3D while also approximating the 3D pose in the camera-coordinate system.

show abstract

“…In [10], the authors predict a full body mesh using the skinned multi-person linear (SMPL) model [25]. First they predict an initial pose with the DMHS detector [26] and refine the prediction using multiple constraints, including reprojection error, a semantic loss involving body part segmentation and matching to ground plane.…”

Section: Https://githubcom/vegesm/depthposementioning

confidence: 99%

“…For example, detecting hand-shakes, object manipulation and passing all require more information than the root-relative pose. To our knowledge, the only solution for absolute pose estimation is finding an optimal translation vector that minimizes the reprojection error [9], [10]. The search for the optimal translation is performed as a This work was completed in the ELTE Institutional Excellence Program (1783-3/2018/FEKUTSRAT) supported by the Hungarian Ministry of Human Capacities.…”

Section: Introductionmentioning

confidence: 99%

Absolute Human Pose Estimation with Depth Prediction Network

Veges

Lörincz

2019

2019 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

The common approach to 3D human pose estimation is predicting the body joint coordinates relative to the hip. This works well for a single person but is insufficient in the case of multiple interacting people. Methods predicting absolute coordinates first estimate a root-relative pose then calculate the translation via a secondary optimization task. We propose a neural network that predicts joints in a camera centered coordinate system instead of a root-relative one. Unlike previous methods, our network works in a single step without any postprocessing. Our network beats previous methods on the MuPoTS-3D dataset and achieves state-of-the-art results.

show abstract

Monocular 3D Pose and Shape Estimation of Multiple People in Natural Scenes: The Importance of Multiple Scene Constraints

Cited by 295 publications

References 29 publications

3D Human Body Reconstruction from a Single Image via Volumetric Regression

3D Human Body Reconstruction from a Single Image via Volumetric Regression

Multi-Person 3D Human Pose Estimation from Monocular Images

Absolute Human Pose Estimation with Depth Prediction Network

Contact Info

Product

Resources

About