Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields

Cao, Zhe; Simon, Tomas; Wei, Shih-En; Sheikh, Yaser

doi:10.48550/arxiv.1611.08050

Cited by 102 publications

(143 citation statements)

References 22 publications

Supporting

Mentioning

131

Contrasting

Order By: Relevance

“…Bottom-up methods: Bottom-up methods [3,7,14,15,27,31] detect identity-free instance agnostic body joints for all the persons in an image and then group them into full-body keypoints. This enables bottom-up methods to be faster and more capable of achieving real-time pose estimation.…”

Section: Related Workmentioning

confidence: 99%

“…Top-down methods [5,11,30,36,37,38,39] take as input an image region within a bounding box, generally the output of a human detector, and reduce the problem to the simpler task of single human pose estimation. Bottom-up methods [3,19,27,29], in contrast, start by independently localizing keypoints in the entire image, followed by grouping them into 2D human pose instances.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Multi-Instance Pose Networks: Rethinking Top-Down Pose Estimation

Khirodkar¹,

Chari²,

Agrawal³

et al. 2021

Preprint

View full text Add to dashboard Cite

A key assumption of top-down human pose estimation approaches is their expectation of having a single person present in the input bounding box. This often leads to failures in crowded scenes with occlusions. We propose a novel solution to overcome the limitations of this fundamental assumption. Our Multi-Hypothesis Pose Network (MHPNet) allows for predicting multiple 2D poses within a given bounding box. We introduce a Multi-Hypothesis Attention Block (MHAB) that can adaptively modulate channel-wise feature responses for each hypothesis and is parameter efficient. We demonstrate the efficacy of our approach by evaluating on COCO, CrowdPose and OCHuman datasets. Specifically, we achieve 70.0 AP on CrowdPose and 42.5 AP on OCHuman test sets, a significant improvement of 2.4 AP and 6.5 AP over prior art, respectively. When using ground truth bounding boxes for inference, MHPNet achieves an improvement of 0.7 AP on COCO, 0.9 AP on CrowdPose, and 9.1 AP on OCHuman validation sets compared to HRNet. Interestingly, when fewer, high confidence bounding boxes are used, HRNet's performance degrades (by 5 AP) on OCHuman, whereas MHPNet maintains a relatively stable performance (drop of 1 AP) for the same inputs.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Multi-Instance Pose Networks: Rethinking Top-Down Pose Estimation

Khirodkar¹,

Chari²,

Agrawal³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Specifically, if all the front-seat humans (passengers and the driver) are seated inposition, the clip was given a positive label, while if any human is seated OOP (based on [106], [107]) for the entire 3s, the clip was labeled as negative. Next, we used Open Pose [108] on each clip to extract a sequence of poses for every person. Our final Dash-Cam-Pose dataset consists of 4875 short videos, 1.06 million poses, of which 310,996 are OOP.…”

Section: Dash-cam-pose Datasetmentioning

confidence: 99%

Generalized One-Class Learning Using Pairs of Complementary Classifiers

Cherian

Wang

2021

Preprint

View full text Add to dashboard Cite

One-class learning is the classic problem of fitting a model to the data for which annotations are available only for a single class. In this paper, we explore novel objectives for one-class learning, which we collectively refer to as Generalized One-class Discriminative Subspaces (GODS). Our key idea is to learn a pair of complementary classifiers to flexibly bound the one-class data distribution, where the data belongs to the positive half-space of one of the classifiers in the complementary pair and to the negative half-space of the other. To avoid redundancy while allowing non-linearity in the classifier decision surfaces, we propose to design each classifier as an orthonormal frame and seek to learn these frames via jointly optimizing for two conflicting objectives, namely: i) to minimize the distance between the two frames, and ii) to maximize the margin between the frames and the data. The learned orthonormal frames will thus characterize a piecewise linear decision surface that allows for efficient inference, while our objectives seek to bound the data within a minimal volume that maximizes the decision margin, thereby robustly capturing the data distribution. We explore several variants of our formulation under different constraints on the constituent classifiers, including kernelized feature maps. We demonstrate the empirical benefits of our approach via experiments on data from several applications in computer vision, such as anomaly detection in video sequences, human poses, and human activities. We also explore the generality and effectiveness of GODS for non-vision tasks via experiments on several UCI datasets, demonstrating state-of-the-art results.

show abstract

“…Here we use Part Affinity Fields (PAFs) method [14] to obtain highly accurate map of human pose estimation in real time. PAFs is flexible and robust since the method detects keypoints firstly and then classifies the keypoints as corresponding body parts.…”

Section: Human Pose Estimationmentioning

confidence: 99%

MsCGAN: Multi-scale Conditional Generative Adversarial Networks for Person Image Generation

Tang

Gui

Bao

et al. 2020

2020 Chinese Control and Decision Conference (CCDC)

View full text Add to dashboard Cite

To synthesize high quality person images with arbitrary poses is challenging. In this paper, we propose a novel Multi-scale Conditional Generative Adversarial Networks (MsCGAN), aiming to convert the input conditional person image to a synthetic image of any given target pose, whose appearance and the texture are consistent with the input image. MsCGAN is a multi-scale adversarial network consisting of two generators and two discriminators. One generator transforms the conditional person image into a coarse image of the target pose globally, and the other is to enhance the detailed quality of the synthetic person image through a local reinforcement network. The outputs of the two generators are then merged into a synthetic, discriminant and high-resolution image. On the other hand, the synthetic image is down-sampled to multiple resolutions as the input to multi-scale discriminator networks. The proposed multi-scale generators and discriminators handling different levels of visual features can benefit to synthesizing high resolution person images with realistic appearance and texture. Experiments are conducted on the Market-1501 and DeepFashion datasets to evaluate the proposed model, and both qualitative and quantitative results demonstrate superior performance of the proposed MsCGAN.

show abstract

Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields

Cited by 102 publications

References 22 publications

Multi-Instance Pose Networks: Rethinking Top-Down Pose Estimation

Multi-Instance Pose Networks: Rethinking Top-Down Pose Estimation

Generalized One-Class Learning Using Pairs of Complementary Classifiers

MsCGAN: Multi-scale Conditional Generative Adversarial Networks for Person Image Generation

Contact Info

Product

Resources

About