PoseFix: Model-Agnostic General Human Pose Refinement Network

Abstract

Multi-person pose estimation from a 2D image is an essential technique for human behavior understanding. In this paper, we propose a human pose refinement network that estimates a refined pose from a tuple of an input image and input pose. The pose refinement was performed mainly through an end-to-end trainable multi-stage architecture in previous methods. However, they are highly dependent on pose estimation models and require careful model design. By contrast, we propose a model-agnostic pose refinement method. According to a recent study, state-of-the-art 2D human pose estimation methods have similar error distributions. We use this error statistics as prior information to generate synthetic poses and use the synthesized poses to train our model. In the testing stage, pose estimation results of any other methods can be input to the proposed method. Moreover, the proposed model does not require code or knowledge about other methods, which allows it to be easily used in the post-processing step. We show that the proposed approach achieves better performance than the conventional multi-stage refinement models and consistently improves the performance of various state-of-the-art pose estimation methods on the commonly used benchmark. The code is available in this https URL 1 .

OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields

“…In the area of object detection, Huang et al [75] show that region-proposed methods (e.g., Faster-rcnn [76]) achieve higher accuracy, while single-shot methods (e.g., YOLO [77], SSD [74]) present higher runtime performance. Analogously in human pose estimation, we observe that top-down approaches also present higher accuracy but lower speed [78] show that refinement over our original work in [3] (by applying a larger cropped image patch) results in a higher accuracy boost than refinement over other top-down approaches. As hardware gets faster and increases its memory, bottom-up methods with higher resolution might be able to reduce the accuracy gap with respect to top-down approaches.…”

FollowMeUp Sports: New Benchmark for 2D Human Keypoint Recognition

“…[28] presents a network to simultaneously output keypoint detections and the corresponding keypoint group assignments. [31] designs a feedback architecture that combining the keypoint results of other pose estimation methods with the original image as the new input to the human pose estimation network. In our analysis we consider 8 state-of-the-art multi-person pose estimation methods, which are listed in Table 2.…”

“…Method AP AP 0.5 AP 0.75 AP M AP L Input Size Runtime Top-down HRNet [7] 0.753 0.925 0.825 0.723 0.803 384x288 0.049 * Xiao [24] 0.723 0.915 0.803 0.695 0.768 256x192 0.110 RMPE [22] 0.735 0.887 0.802 0.693 0.799 320x256 0.298 Bottom-up PAF [9] 0.469 0.737 0.493 0.403 0.561 432x368 0.081 Osokin [10] 0.400 0.659 0.407 0.338 0.494 368x368 0.481 PifPaf [30] 0.630 0.855 0.691 0.603 0.677 401x401 0.202 AE [28] 0.566 0.818 0.618 0.498 0.670 512x512 0.260 PoseFix [31] 0.411 0.647 0.412 0.303 0.559 384x288 0.250 * : without human detection algorithms are lower than top-down methods. After detailed analysis, we find that the numbers of predicted effective keypoints of bottom-up methods are around 10 times less than top-down methods as illustrated in Fig.…”

“…In this dataset, the precision of [10] decreases by 13% in AP 0.5 compared with [9], which indicates that the generality of [10] is also narrowed. We use the results of [9] as the initial poses of [31]. Through pose refinement, [31] improved the pose estimation results by 0.4%.…”

“…We use the results of [9] as the initial poses of [31]. Through pose refinement, [31] improved the pose estimation results by 0.4%.…”