Graph Stacked Hourglass Networks for 3D Human Pose Estimation

Xu, Tianhan; Takano, Wataru

doi:10.1109/cvpr46437.2021.01584

Cited by 144 publications

(63 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They are also superior to RGB data in terms of computational cost. Due to these advantages and the progress of sensors (Zhang 2012) and pose estimation models (Wandt et al 2021;Xu and Takano 2021), various models have been proposed (Yan, Xiong, and Lin 2018;Zhang et al 2020;Cheng et al 2021;Kong, Deng, and Jiang 2021).…”

Section: Related Workmentioning

confidence: 99%

Adversarial Bone Length Attack on Action Recognition

Tanaka

Kera

Kawamoto

2022

AAAI

View full text Add to dashboard Cite

Skeleton-based action recognition models have recently been shown to be vulnerable to adversarial attacks. Compared to adversarial attacks on images, perturbations to skeletons are typically bounded to a lower dimension of approximately 100 per frame. This lower-dimensional setting makes it more difficult to generate imperceptible perturbations. Existing attacks resolve this by exploiting the temporal structure of the skeleton motion so that the perturbation dimension increases to thousands. In this paper, we show that adversarial attacks can be performed on skeleton-based action recognition models, even in a significantly low-dimensional setting without any temporal manipulation. Specifically, we restrict the perturbations to the lengths of the skeleton's bones, which allows an adversary to manipulate only approximately 30 effective dimensions. We conducted experiments on the NTU RGB+D and HDM05 datasets and demonstrate that the proposed attack successfully deceived models with sometimes greater than 90% success rate by small perturbations. Furthermore, we discovered an interesting phenomenon: in our low-dimensional setting, the adversarial training with the bone length attack shares a similar property with data augmentation, and it not only improves the adversarial robustness but also improves the classification accuracy on the original data. This is an interesting counterexample of the trade-off between adversarial robustness and clean accuracy, which has been widely observed in studies on adversarial training in the high-dimensional regime.

show abstract

Section: Related Workmentioning

confidence: 99%

Adversarial Bone Length Attack on Action Recognition

Tanaka

Kera

Kawamoto

2022

AAAI

View full text Add to dashboard Cite

show abstract

“…The probability for a pixel to be the keypoint can be measured by its response in the heatmap. Recently, heatmap-based approaches have achieved the state-of-the-art performance in pose estimation [32,4,34,27]. The coordinates of keypoints are obtained by decoding the heatmaps [25].…”

Section: Related Work and Major Contributionsmentioning

confidence: 99%

“…[4] predicted scale-aware high-resolution heatmaps using multi-resolution aggregation during inference. [34] processed graph-structured features across multi-scale human skeletal representations and proposed a learning approach for multi-level feature learning and heatmap estimation.…”

Section: Related Work and Major Contributionsmentioning

confidence: 99%

Self-Constrained Inference Optimization on Structural Groups for Human Pose Estimation

Kan¹,

Chen²,

Zeng³

et al. 2022

Preprint

View full text Add to dashboard Cite

We observe that human poses exhibit strong group-wise structural correlation and spatial coupling between keypoints due to the biological constraints of different body parts. This group-wise structural correlation can be explored to improve the accuracy and robustness of human pose estimation. In this work, we develop a self-constrained prediction-verification network to characterize and learn the structural correlation between keypoints during training. During the inference stage, the feedback information from the verification network allows us to perform further optimization of pose prediction, which significantly improves the performance of human pose estimation. Specifically, we partition the keypoints into groups according to the biological structure of human body. Within each group, the keypoints are further partitioned into two subsets, high-confidence base keypoints and low-confidence terminal keypoints. We develop a self-constrained prediction-verification network to perform forward and backward predictions between these keypoint subsets. One fundamental challenge in pose estimation, as well as in generic prediction tasks, is that there is no mechanism for us to verify if the obtained pose estimation or prediction results are accurate or not, since the ground truth is not available. Once successfully learned, the verification network serves as an accuracy verification module for the forward pose prediction. During the inference stage, it can be used to guide the local optimization of the pose estimation results of lowconfidence keypoints with the self-constrained loss on high-confidence keypoints as the objective function. Our extensive experimental results on benchmark MS COCO and CrowdPose datasets demonstrate that the proposed method can significantly improve the pose estimation results.

show abstract

“…Given the 3D supervision, the spatial relationship can be directly learned via supervised learning [6,20,55,59]. Various representations have been proposed to effectively encode the spatial relationship such as volumetric representation [43], graph structure [4,11,71,76], transformer architecture [31,34,74], compact designs for realtime reconstruction [37,38], and inverse kinematics [30]. These supervised learning approaches that rely on the 3D ground truth supervision, however, show limited generalization to images of out-of-distribution scenes and poses due to the domain gap.…”

Section: Related Workmentioning

confidence: 99%

PoseKernelLifter: Metric Lifting of 3D Human Pose using Sound

Yang

Fan²,

Isler³

et al. 2021

Preprint

View full text Add to dashboard Cite

Single view image (b) Audio (c) Metric 3D reconstruction 1m 1m * Work performed as a member of Samsung AI Center NY.a multi-stage 3D CNN that fuses audio and visual signals and learns to reconstruct 3D pose in a metric scale. We show that our multi-modal method produces accurate metric reconstruction in realworld scenes, which is not possible with state-of-the-art lifting approaches including parametric mesh regression and depth regression.

show abstract

Graph Stacked Hourglass Networks for 3D Human Pose Estimation

Cited by 144 publications

References 44 publications

Adversarial Bone Length Attack on Action Recognition

Adversarial Bone Length Attack on Action Recognition

Self-Constrained Inference Optimization on Structural Groups for Human Pose Estimation

PoseKernelLifter: Metric Lifting of 3D Human Pose using Sound

Contact Info

Product

Resources

About