Multi-Instance Pose Networks: Rethinking Top-Down Pose Estimation

Khirodkar, Rawal; Chari, Visesh; Agrawal, Amit; Tyagi, Ambrish

doi:10.1109/iccv48922.2021.00311

Cited by 58 publications

(20 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This blurry dataset was created by applying a randomized rotational and linear blur to images from the previous testing set. 8 We then conducted a similar test on this dataset with mostly the same pipeline settings and evaluation criteria as before with only a few minor changes. Firstly, we set the pipeline to deblur the image prior to finding areas of interest.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

SHOP: A Deep Learning Based Pipeline for near Real-Time Detection of Small Handheld Objects Present in Blurry Video

Ganguly,

Gandhi,

et al. 2022

Preprint

View full text Add to dashboard Cite

While prior works have investigated and developed computational models capable of object detection, models still struggle to reliably interpret images with motion blur and small objects. Moreover, none of these models are specifically designed for handheld object detection. In this work, we present SHOP (Small Handheld Object Pipeline), a pipeline that reliably and efficiently interprets blurry images containing handheld objects. The specific models used in each stage of the pipeline are flexible and can be changed based on performance requirements. First, images are deblurred and then run through a pose detection system where areas-of-interest are proposed around the hands of any people present. Next, object detection is performed on the images by a single-stage object detector. Finally, the proposed areas-of-interest are used to filter out low confidence detections. Testing on a handheld subset of Microsoft Common Objects in Context (MS COCO) demonstrates that this 3 stage process results in a 70 percent decrease in false positives while only reducing true positives by 17 percent in its strongest configuration. We also present a subset of MS COCO consisting solely of handheld objects that can be used to continue the development of handheld object detection methods. https://github.com/spider-sense/SHOP

show abstract

Section: Resultsmentioning

confidence: 99%

“…Popular top-down approaches in pose estimation are SimDR [5], UDP-Pose-PSA [6], OmniPose [7], and HRNet-W48 [8]. Popular bottom-up approaches are Disentangled Keypoint Regression (DEKR) [9], SimplePose [10], and OpenPose (2017) [4].…”

Section: B 2d Pose Estimationmentioning

confidence: 99%

SHOP: A Deep Learning Based Pipeline for near Real-Time Detection of Small Handheld Objects Present in Blurry Video

Ganguly,

Gandhi,

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In the literature, there are only few methods have reported results on this challenging dataset. Compared to the current best method MIPNet [14], our SCIO method has improved the pose estimation accuracy by up to 1.5%, which is quite significantly.…”

Section: Comparison To State Of the Artmentioning

confidence: 90%

Self-Constrained Inference Optimization on Structural Groups for Human Pose Estimation

Kan¹,

Chen²,

Zeng³

et al. 2022

Preprint

View full text Add to dashboard Cite

We observe that human poses exhibit strong group-wise structural correlation and spatial coupling between keypoints due to the biological constraints of different body parts. This group-wise structural correlation can be explored to improve the accuracy and robustness of human pose estimation. In this work, we develop a self-constrained prediction-verification network to characterize and learn the structural correlation between keypoints during training. During the inference stage, the feedback information from the verification network allows us to perform further optimization of pose prediction, which significantly improves the performance of human pose estimation. Specifically, we partition the keypoints into groups according to the biological structure of human body. Within each group, the keypoints are further partitioned into two subsets, high-confidence base keypoints and low-confidence terminal keypoints. We develop a self-constrained prediction-verification network to perform forward and backward predictions between these keypoint subsets. One fundamental challenge in pose estimation, as well as in generic prediction tasks, is that there is no mechanism for us to verify if the obtained pose estimation or prediction results are accurate or not, since the ground truth is not available. Once successfully learned, the verification network serves as an accuracy verification module for the forward pose prediction. During the inference stage, it can be used to guide the local optimization of the pose estimation results of lowconfidence keypoints with the self-constrained loss on high-confidence keypoints as the objective function. Our extensive experimental results on benchmark MS COCO and CrowdPose datasets demonstrate that the proposed method can significantly improve the pose estimation results.

show abstract

“…Single-stage Human Pose Estimation. Methods for 2D human pose estimation generally fall into two categories: single-stage methods [4,6,11,15,22,38,40], and two-stage methods [5,21,33,39,49,59,60]. Two-stage methods detect the people in an image using an off-the-shelf person detector (e.g., Faster R-CNN [47], YOLOv3 [46], etc.)…”

Section: Related Workmentioning

confidence: 99%

“…Top-down methods (e.g., Alpha-Pose [9] and Mask-RCNN [15]) typically perform worse on CrowdPose as person detections with multiple people are problematic for the subsequent keypoint estimation. MIP-Net [21] is an exception, as the authors explicitly address the aforementioned problem in their top-down approach.…”

Section: Crowdposementioning

confidence: 99%

Rethinking Keypoint Representations: Modeling Keypoints and Poses as Objects for Multi-Person Human Pose Estimation

McNally¹,

Vats²,

Wong³

et al. 2021

Preprint

View full text Add to dashboard Cite

In keypoint estimation tasks such as human pose estimation, heatmap-based regression is the dominant approach despite possessing notable drawbacks: heatmaps intrinsically suffer from quantization error and require excessive computation to generate and post-process. Motivated to find a more efficient solution, we propose a new heatmapfree keypoint estimation method in which individual keypoints and sets of spatially related keypoints (i.e., poses) are modeled as objects within a dense single-stage anchorbased detection framework. Hence, we call our method KAPAO (pronounced "Ka-Pow!") for Keypoints And Poses As Objects. We apply KAPAO to the problem of single-stage multi-person human pose estimation by simultaneously detecting human pose objects and keypoint objects and fusing the detections to exploit the strengths of both object representations. In experiments, we observe that KAPAO is significantly faster and more accurate than previous methods, which suffer greatly from heatmap post-processing. Moreover, the accuracy-speed trade-off is especially favourable in the practical setting when not using test-time augmentation. Our large model, KAPAO-L, achieves an AP of 70.6 on the Microsoft COCO Keypoints validation set without testtime augmentation while being 2.5× faster than the next best single-stage model, whose accuracy is 4.0 AP less. Furthermore, KAPAO excels in the presence of heavy occlusion. On the CrowdPose test set, KAPAO-L achieves new state-of-the-art accuracy for a single-stage method with an AP of 68.9.

show abstract

Multi-Instance Pose Networks: Rethinking Top-Down Pose Estimation

Cited by 58 publications

References 29 publications

SHOP: A Deep Learning Based Pipeline for near Real-Time Detection of Small Handheld Objects Present in Blurry Video

SHOP: A Deep Learning Based Pipeline for near Real-Time Detection of Small Handheld Objects Present in Blurry Video

Self-Constrained Inference Optimization on Structural Groups for Human Pose Estimation

Rethinking Keypoint Representations: Modeling Keypoints and Poses as Objects for Multi-Person Human Pose Estimation

Contact Info

Product

Resources

About