VIBE: Video Inference for Human Body Pose and Shape Estimation

Kocabas, Muhammed; Athanasiou, Nikos; Black, Michael J.

doi:10.48550/arxiv.1912.05656

Cited by 14 publications

(18 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We are particularly interested in integrating the renderer with systems that infer SMPL bodies from images (e.g. [22,28,27]) to enable an end-to-end system for body image generation trained from images in the wild.…”

Section: Discussionmentioning

confidence: 99%

SMPLpix: Neural Avatars from 3D Human Models

Prokudin¹,

Black²,

Romero³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Recent advances in deep generative models have led to an unprecedented level of realism for synthetically generated images of humans. However, one of the remaining fundamental limitations of these models is the ability to flexibly control the generative process, e.g. change the camera and human pose while retaining the subject identity. At the same time, deformable human body models like SMPL [35] and its successors provide full control over pose and shape, but rely on classic computer graphics pipelines for rendering. Such rendering pipelines require explicit mesh rasterization that (a) does not have the potential to fix artifacts or lack of realism in the original 3D geometry and (b) until recently, were not fully incorporated into deep learning frameworks. In this work, we propose to bridge the gap between classic geometry-based rendering and the latest generative networks operating in pixel space by introducing a neural rasterizer, a trainable neural network module that directly "renders" a sparse set of 3D mesh vertices as photorealistic images, avoiding any hardwired logic in pixel colouring and occlusion reasoning. We train our model on a large corpus of human 3D models and corresponding real photos, and show the advantage over conventional differentiable renderers both in terms of the level of photorealism and rendering efficiency.

show abstract

Section: Discussionmentioning

confidence: 99%

SMPLpix: Neural Avatars from 3D Human Models

Prokudin¹,

Black²,

Romero³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The weight for this term is set to be 50× the weight of the prior, as we expect a lower variance for pose dynamics. We compare our method with the recent work of [18], showing the results in table 3. As in the static setting, these are also the lowest errors reported so far.…”

Section: Methodsmentioning

confidence: 99%

Weakly Supervised 3D Human Pose and Shape Reconstruction with Normalizing Flows

Zanfir¹,

Bazavan²,

Xu³

et al. 2020

Preprint

View full text Add to dashboard Cite

Monocular 3D human pose and shape estimation is challenging due to the many degrees of freedom of the human body and the difficulty to acquire training data for large-scale supervised learning in complex visual scenes. In this paper we present practical semi-supervised and self-supervised models that support training and good generalization in real-world images and video. Our formulation is based on kinematic latent normalizing flow representations and dynamics, as well as differentiable, semantic body part alignment loss functions that support selfsupervised learning. In extensive experiments using 3D motion capture datasets like CMU, Human3.6M, 3DPW, or AMASS, as well as image repositories like COCO, we show that the proposed methods outperform the state of the art, supporting the practical construction of an accurate family of models based on large-scale training with diverse and incompletely labeled image and video data.

show abstract

“…A step is considered valid only if the heel of one foot touches the toe of another foot. Then subject's 3D body key-points were extracted using VIBE system [9]. VIBE (Video Inference for Body Pose and Shape Estimation) is a video pose and shape estimation method that predicts the parameters of SMPL body model for each frame of an input video.…”

Section: Dataset Descriptionmentioning

confidence: 99%

“…In this work (Figure 2), the focus of the automated assessment system is on the "Tandem gait" task which is part of a larger system called ATEC-Activated Test of Embodied Cognition [1,3,17]. A dataset has been created with 27 children performing the gait In order to automatically evaluate subject's performance, first VIBE [9] human pose estimation system was used to extract 3D body key-points. Then a deep learning based model was trained to classify subject's steps as valid or invalid.…”

mentioning

confidence: 99%

Automated system to measure Tandem Gait to assess executive functions in children

Zadeh,

Babu,

Jaiswal

et al. 2020

Preprint

View full text Add to dashboard Cite

As mobile technologies have become ubiquitous in recent years, computer-based cognitive tests have become more popular and efficient. In this work, we focus on assessing motor function in children by analyzing their gait movements. Although there has been a lot of research on designing automated assessment systems for gait analysis, most of these efforts use obtrusive wearable sensors for measuring body movements. We have devised a computer visionbased assessment system that only requires a camera which makes it easier to employ in school or home environments. A dataset has been created with 27 children performing the test. Furthermore in order to improve the accuracy of the system, a deep learning based model was pre-trained on NTU-RGB+D 120 dataset and then it was fine-tuned on our gait dataset. The results highlight the efficacy of proposed work for automating the assessment of children's performances by achieving 76.61% classification accuracy. CCS CONCEPTS• Computing methodologies → Activity recognition and understanding.

show abstract

VIBE: Video Inference for Human Body Pose and Shape Estimation

Cited by 14 publications

References 45 publications

SMPLpix: Neural Avatars from 3D Human Models

SMPLpix: Neural Avatars from 3D Human Models

Weakly Supervised 3D Human Pose and Shape Reconstruction with Normalizing Flows

Automated system to measure Tandem Gait to assess executive functions in children

Contact Info

Product

Resources

About