CrowdPose: Efficient Crowded Scenes Pose Estimation and a New Benchmark

Abstract

Multi-person pose estimation is fundamental to many computer vision tasks and has made significant progress in recent years. However, few previous methods explored the problem of pose estimation in crowded scenes while it remains challenging and inevitable in many scenarios. Moreover, current benchmarks cannot provide an appropriate evaluation for such cases. In this paper, we propose a novel and efficient method to tackle the problem of pose estimation in the crowd and a new dataset to better evaluate algorithms. Our model consists of two key components: joint-candidate single person pose estimation (SPPE) and global maximum joints association. With multipeak prediction for each joint and global association using graph model, our method is robust to inevitable interference in crowded scenes and very efficient in inference. The proposed method surpasses the state-of-the-art methods on CrowdPose dataset by 5.2 mAP and results on MSCOCO dataset demonstrate the generalization ability of our method. Source code and dataset will be made publicly available.

Human Pose Estimation for Real-World Crowded Scenarios

“…Each of them comes with different advantages and disadvantages. Although bottom-up methods seem to be better suited for crowded scenes since they process whole images and their runtime is thus less dependent of the actual number of persons, literature shows that top-down methods like [6] and [11] perform comparably well on publicly available datasets.…”

“…However, some recent works focus on this subject. Li et al [6] introduce a new benchmark for evaluating pose estimation methods for this problem and proposed a method where human bounding box proposals obtained by human detector are fed into joint-candidate single person pose estimator (JCSPPE). JC-SPPE locates the joint candidates with different response scores on the heatmap.…”

“…Using Grand Theft Auto V, a widely known computer game with an active modding community, they created a synthetic dataset of humans annotated with highly accurate keypoint information along-side with a tool that allows to easily generate own synthetic datasets. Both, [3] and [6] proposed interesting datasets, which we used for our experiments, whereas [3] is more similar to surveillance situations than the dataset introduced by [6].…”

“…Another goal in creating an extension for the JTA dataset was to increase the share of highly crowded scenarios. In order to gauge the "crowdedness" of the datasets we utilized the CrowdIndex proposed by Li et al [6]. The different CrowdIndex distributions are presented in Fig.…”

Learning Compositional Neural Information Fusion for Human Parsing

“…Hierarchical/Graphical Models in Computer Vision: Hierarchical/graphical models are powerful for building structured representations, which can reflect task-specific relations and constraints. From early distributional semantic models, part-based models [16,17], MRF/CRF [31], And-Or grammar model [59], to deep structural networks [30,15], graph neural networks [20], trainable CRF [79], etc., hierarchical/graphical models have found applications in a wide variety of core computer vision tasks, such as object recognition [55], human parsing [40,41,81], pose estimation [34,66,61,68,35], visual dialog etc., to the extent that they are now ubiquitous in the field. Inspired by their general success, we leverage structural information to design our approach.…”