Multi-Instance Pose Networks: Rethinking Top-Down Pose Estimation

Abstract

A key assumption of top-down human pose estimation approaches is their expectation of having a single person present in the input bounding box. This often leads to failures in crowded scenes with occlusions. We propose a novel solution to overcome the limitations of this fundamental assumption. Our Multi-Hypothesis Pose Network (MHPNet) allows for predicting multiple 2D poses within a given bounding box. We introduce a Multi-Hypothesis Attention Block (MHAB) that can adaptively modulate channel-wise feature responses for each hypothesis and is parameter efficient. We demonstrate the efficacy of our approach by evaluating on COCO, CrowdPose and OCHuman datasets. Specifically, we achieve 70.0 AP on CrowdPose and 42.5 AP on OCHuman test sets, a significant improvement of 2.4 AP and 6.5 AP over prior art, respectively. When using ground truth bounding boxes for inference, MHPNet achieves an improvement of 0.7 AP on COCO, 0.9 AP on CrowdPose, and 9.1 AP on OCHuman validation sets compared to HRNet. Interestingly, when fewer, high confidence bounding boxes are used, HRNet's performance degrades (by 5 AP) on OCHuman, whereas MHPNet maintains a relatively stable performance (drop of 1 AP) for the same inputs.

A Lightweight Multi-Person Pose Estimation Scheme Based on Jetson Nano

“…Bertasius et al (2019) extend from images to videos and propose a method for learning pose warping on sparsely labeled videos. Khirodkar et al (2021) offer a Multi-Instance Pose Network (MIPNet) that predicts multiple 2D pose instances within a bounding box. It can overcome the limitations of crowded scenes with occlusions.…”

Occluded Human Mesh Recovery

“…In contrast, datasets like MPI-INF-3DHP [46], PanopticStudio [21] and 3DPW [61] contain multi-person annotations but have limited person-person occlusion -less than 27% of all annotations have crowding (at IoU 0.5). Although previous methods [24,29,30,32,35,36,67] leverage 2D keypoint annotations from datasets like COCO [39], MPII [1], LSP-Extended [20], the 2D datasets are also known to contain similar biases [28,53,69]. These biases have affected critical design decisions in state-of-the-art methods which lead to poor generalization under heavy occlusion [19,58].…”