YOLO-Pose: Enhancing YOLO for Multi Person Pose Estimation Using Object Keypoint Similarity Loss

Maji, Debapriya; Nagori, Soyeb; Mathew, Manu; Poddar, Deepak

doi:10.1109/cvprw56347.2022.00297

Cited by 126 publications

(62 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This section examines the transferability of the proposed adversarial attack method to other pose estimation models, which is similar to the characteristics of black-box adversarial attacks. We select three other typical pose estimation models, which are YOLO-Pose, 40 TransPose, 41 and AlphaPose, 36 to prove the transferability of our proposed method.…”

Section: Transferability To Other Pose Estimation Modelsmentioning

confidence: 99%

Adversarial attack on human pose estimation network

Zhang,

Huang,

Liu

et al. 2024

J. Electron. Imag.

View full text Add to dashboard Cite

Real-time human pose estimation (HPE) using convolutional neural networks (CNN) is critical for enabling machines to better understand human beings based on images and videos, and for assisting supervisors in identifying human behavior. However, CNN-based systems are susceptible to adversarial attacks, and the attacks specifically targeting HPE have received little attention. We present a gradient-based adversarial example generation method, named AdaptiveFool, which is designed to effectively perform a keypoints-invisible attack against OpenPose by aggregating the loss function of human keypoints and generating adaptive adversarial perturbations. In addition, we introduce an object-oriented perturbation generation method during the AdaptiveFool process to eliminate background perturbations. Our proposed method adapts the adversarial perturbations and generates object-oriented perturbations. On COCO 2017 datasets, our method achieves 6.3% mean average precision on OpenPose. This research provides inspiration for future work on developing efficient and effective adversarial example defense methods for HPE.

show abstract

Section: Transferability To Other Pose Estimation Modelsmentioning

confidence: 99%

Adversarial attack on human pose estimation network

Zhang,

Huang,

Liu

et al. 2024

J. Electron. Imag.

View full text Add to dashboard Cite

show abstract

“…We propose to predict image-space keypoints normalized by the scale. The idea is related to human/object keypoints prediction with area-normalization [34], [35] or object-sizenormalization [36], but for grasp pose estimation problem we normalize with pose proximity to camera center to introduce scale invariance. Specifically, we scale the offset which determines the proximity of keypoints.…”

Section: B Pose Estimation With Scale-normalized Keypointmentioning

confidence: 99%

Video based basketball shooting prediction and pose suggestion system

Chen¹,

Chang²,

Lin³

et al. 2023

Multimed Tools Appl

View full text Add to dashboard Cite

We propose a new 6-DoF grasp pose synthesis approach from 2D/2.5D input based on keypoints. Keypointbased grasp detector from image input has demonstrated promising results in the previous study, where the additional visual information provided by color images compensates for the noisy depth perception. However, it relies heavily on accurately predicting the location of keypoints in the image space. In this paper, we devise a new grasp generation network that reduces the dependency on precise keypoint estimation. Given an RGB-D input, our network estimates both the grasp pose from keypoint detection as well as scale towards the camera. We further re-design the keypoint output space in order to mitigate the negative impact of keypoint prediction noise to Perspectiven-Point (PnP) algorithm. Experiments show that the proposed method outperforms the baseline by a large margin, validating the efficacy of our approach. Finally, despite trained on simple synthetic objects, our method demonstrate sim-to-real capacity by showing competitive results in real-world robot experiments.Code is available at: https://github.com/ivalab/KGN.

show abstract

“…where p denotes the index of the spatial location in the bias map; i denotes the index of the category for p th location; d pi denotes the Euclidean distance between the predicted and labelled location for keypoint determined by p and i, where predicted location is (p − o pred pi ) and labelled location is (p − o gt pi ); S p denotes the scale factor of the person instance p, numerically equals to the area of person instance p; 𝜎 i denotes the standard deviation for the i th keypoint over on entire dataset, reflecting the difficulty of labelling the keypoint; h p denotes the confidence that position p is the body centroid and numerically equals to the value of position p on the centroid heat map; δ denotes the function that selects a particular h p for calculation. Unlike the OKS loss proposed in PETR [8] and YoloPose [26], we apply different weights to the L OKS at different position p. The current optimization objectives of L OKS are in line with the evaluation metrics, and with the optimization of L OKS , the network can achieve improved performance.…”

Section: Keypoint Similarity Lossmentioning

confidence: 99%

“…Work [25] set different weights for keypoints in the calculation of the loss to distinguish the impact of keypoints on pose estimation. YoloPose [26] has addressed the shortcomings of non‐end‐to‐end training of heatmap‐based pose estimation methods by proposing OKS loss. These works demonstrate the importance of using scientific loss functions to improve the performance of pose estimation networks.…”

Section: Related Workmentioning

confidence: 99%

LiteDEKR: End‐to‐end lite 2D human pose estimation network

Tian

et al. 2023

IET Image Processing

View full text Add to dashboard Cite

The 2D human pose estimation plays an important role in human‐computer interaction and action recognition. Although the method based on high‐resolution network has superior performance, there is still room for improvement in terms of speed and lightweight. Here, a LiteDEKR, a 2D pose estimation method that combines lightweight and accuracy, is proposed by designing a lightweight network based on DEKR and constructing two scientifically valid loss functions. The method, constructs a multi‐instance bias regression loss that matches the true distribution of keypoint bias, improves the accuracy of bias regression, and constructs a keypoint similarity loss with the object keypoint similarity index of keypoints as the optimization objective to achieve end‐to‐end training of the network. In addition, this paper has designed a lightweight DEKR, using LitePose as the backbone network. With the optimization of the above two loss functions, LiteDEKR not only achieves lightweight but also has high accuracy. Comparative experiments on the COCO and CrowdPose datasets show that compared to the current state‐of‐the‐art Contextual Instance Decoupling, LiteDEKR achieves a similar accuracy with only 10% of its network complexity. It also shows better robustness to low‐resolution input images.

show abstract

YOLO-Pose: Enhancing YOLO for Multi Person Pose Estimation Using Object Keypoint Similarity Loss

Cited by 126 publications

References 19 publications

Adversarial attack on human pose estimation network

Adversarial attack on human pose estimation network

Video based basketball shooting prediction and pose suggestion system

LiteDEKR: End‐to‐end lite 2D human pose estimation network

Contact Info

Product

Resources

About