Test-Time Adaptation for 3D Human Pose Estimation

Amin, Sikandar; Müller, Philipp; Bulling, Andreas; Andriluka, Mykhaylo

doi:10.1007/978-3-319-11752-2_20

Cited by 11 publications

(13 citation statements)

References 18 publications

(50 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This problem corresponds to optimizing the energy function in Eq. (2). Note that using the optimization algorithm of [23] would require to construct a 3D state space that includes all 2D positions back-projected to 3D (amounting to the number of views multiplied by the size of the images) augmented with extra nodes for each back-projected node to account for occlusions.…”

Section: Multi-view Rgb-d Optimizationmentioning

confidence: 99%

“…In a multi-view RGB system, correspondences across views are traditionally established by relying on appearance similarity and triangulation [6,14,2,7], which is unreliable in OR environments containing many surfaces that are visually similar. Instead, the depth data enables us to backproject points to 3D and is not affected by the visual appearance of the surfaces in the scene [23].…”

Section: Introductionmentioning

confidence: 99%

“…Most work on multi-view human pose estimation focuses on single-person scenarios in controlled environments to reduce the ambiguity of data association by relying on background subtraction or exemplar-based approaches [14,18,31]. In order to be robust to cluttered background, part-based approaches are used to generate part hypotheses per view and then estimate the body configuration in 2D [1,2] or in 3D [8].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Multi-view RGB-D Approach for Human Pose Estimation in Operating Rooms

Kadkhodamohammadi

Gangi

Mathelin

et al. 2017

2017 IEEE Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

Many approaches have been proposed for human pose estimation in single and multi-view RGB images. However, some environments, such as the operating room, are still very challenging for state-of-the-art RGB methods. In this paper, we propose an approach for multi-view 3D human pose estimation from RGB-D images and demonstrate the benefits of using the additional depth channel for pose refinement beyond its use for the generation of improved features. The proposed method permits the joint detection and estimation of the poses without knowing a priori the number of persons present in the scene. We evaluate this approach on a novel multi-view RGB-D dataset acquired during live surgeries and annotated with ground truth 3D poses.

show abstract

Section: Multi-view Rgb-d Optimizationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Multi-view RGB-D Approach for Human Pose Estimation in Operating Rooms

Kadkhodamohammadi

Gangi

Mathelin

et al. 2017

2017 IEEE Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

show abstract

“…But, these single-view datasets are lacking 3D annotations, contrary to multi-view datasets, which often come with accurate 3D body poses. As these are however much simpler for the task of 2D pose estimation [11,28], researchers have proposed methods to jointly use both single-and multi-view datasets in order to construct more robust 3D pose estimation models from multiple views [3,7]. Changes in camera setups however require the retraining of the model on training data from the same camera setup.…”

Section: Introductionmentioning

confidence: 99%

A generalizable approach for multi-view 3D human pose regression

Kadkhodamohammadi¹,

Padoy²

2018

Preprint

View full text Add to dashboard Cite

Despite the significant improvement in the performance of monocular pose estimation approaches and their ability to generalize to unseen environments, multi-view approaches are often lagging behind in terms of accuracy and are specific to certain datasets. This is mainly due to the fact that (1) contrary to real world single-view datasets, multi-view datasets are often captured in controlled environments to collect precise 3D annotations, which do not cover all real world challenges, and (2) the model parameters are learned for specific camera setups. To alleviate these problems, we propose a two-stage approach to detect and estimate 3D human poses, which separates single-view pose detection from multi-view 3D pose estimation. This separation enables us to utilize each dataset for the right task, i.e. single-view datasets for constructing robust pose detection models and multi-view datasets for constructing precise multi-view 3D regression models. In addition, our 3D regression approach only requires 3D pose data and its projections to the views for building the model, hence removing the need for collecting annotated data from the test setup. Our approach can therefore be easily generalized to a new environment by simply projecting 3D poses into 2D during training according to the camera setup used at test time. As 2D poses are collected at test time using a single-view pose detector, which might generate inaccurate detections, we model its characteristics and incorporate this information during training. We demonstrate that incorporating the detector's characteristics is important to build a robust 3D regression model and that the resulting regression model generalizes well to new multi-

show abstract

“…Related work. The fact that human appearance tends to stay unchanged through videos has been used in the past to aid pose estimation [1,36,41]. Ramanan et al [36] train discriminative body part detectors by first detecting 'easy' poses (such as a 'scissors' walking pose) and then using the appearance learnt from these poses to track the remaining video with a pictorial structure model.…”

Section: Introductionmentioning

confidence: 99%

Personalizing Human Video Pose Estimation

Charles

Pfister

Magee

et al. 2016

2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

We propose a personalized ConvNet pose estimator that automatically adapts itself to the uniqueness of a person's appearance to improve pose estimation in long videos.We make the following contributions: (i) we show that given a few high-precision pose annotations, e.g. from a generic ConvNet pose estimator, additional annotations can be generated throughout the video using a combination of image-based matching for temporally distant frames, and dense optical flow for temporally local frames; (ii) we develop an occlusion aware self-evaluation model that is able to automatically select the high-quality and reject the erroneous additional annotations; and (iii) we demonstrate that these high-quality annotations can be used to fine-tune a ConvNet pose estimator and thereby personalize it to lock on to key discriminative features of the person's appearance. The outcome is a substantial improvement in the pose estimates for the target video using the personalized ConvNet compared to the original generic ConvNet.Our method outperforms the state of the art (including top ConvNet methods) by a large margin on three standard benchmarks, as well as on a new challenging YouTube video dataset. Furthermore, we show that training from the automatically generated annotations can be used to improve the performance of a generic ConvNet on other benchmarks.

show abstract

Test-Time Adaptation for 3D Human Pose Estimation

Cited by 11 publications

References 18 publications

A Multi-view RGB-D Approach for Human Pose Estimation in Operating Rooms

A Multi-view RGB-D Approach for Human Pose Estimation in Operating Rooms

A generalizable approach for multi-view 3D human pose regression

Personalizing Human Video Pose Estimation

Contact Info

Product

Resources

About