Forward-reverse adaptive graph convolutional networks for skeleton-based action recognition

Hu, Zesheng; Pan, Zihao; Wang, Qiang; Yu, Lei; Fei, Shumin

doi:10.1016/j.neucom.2021.12.054

Cited by 14 publications

(7 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Year CS(%) CV(%) GCA-LSTM [17] 2018 74.3 82.8 ST-GCN [25] 2018 81.5 88.3 SR-TSL [15] 2018 84.8 92.4 HCN [69] 2018 86.5 91.1 DPRL+GCNN [35] 2018 83.5 89.8 AS-GCN [50] 2019 86.8 94.2 AGC-LSTM [70] 2019 89.2 95.0 2s-AGCN [26] 2019 88.5 95.1 DGNN [39] 2019 89.9 96.1 BAGCN [61] 2019 90.3 96.3 STGR-GCN [60] 2019 86.9 92.3 TS-SAN [71] 2020 87.2 92.7 2s-AAGCN+TEM [47] 2020 88.7 95.8 SGN [72] 2020 89.0 94.5 2s-Shift-GCN [29] 2020 89.7 96 GCN-NAS [48] 2020 89.4 95.7 2s-AAGCN [27] 2020 89.4 96.0 MS-AAGCN [27] 2020 90.0 96.2 CGCN [63] 2020 90.3 96.4 4s-Shift-GCN [29] 2020 90.7 96.5 MS-AAGCN+TEM [47] 2020 91 96.5 Dynamic GCN [30] 2020 91.5 96 MS-G3D [28] 2020 91.5 96.2 PR-GCN [64] 2021 85.2 91.7 RA-GCN [49] 2021 87.3 93.6 SEFN(Base) [31] 2021 89.2 95.8 ST-TR [36] 2021 89.9 96.1 SEFN(Att) [31] 2021 90.2 96.1 ST-TR-agcn [36] 2021 90.3 96.3 MSTGNN [73] 2021 91.3 95.5 PB-GCN [65] 2022 83.8 91.3 PeGCN [66] 2022 85.6 93.4 LAGA [51] 2022 87.1 93.2 TE-GCN [68] 2022 88.7 95.4 EGCN [67] 2022 89.1 95.5 Graph2Net [32] 2022 90.1 96 Sybio-GNN [33] 2022 90.1 95.4 CD-GCN [34] 2022 90.1 96.5 Ta-CNN [24] 2022 90.7 95.1 FR-AGCN [52] 2022 90. is c...…”

Section: Methodsmentioning

confidence: 99%

“…Year CS(%) CV(%) GCA-LSTM [17] 2018 61.2 63.3 RotClips+MTCNN [74] 2018 62.2 61.8 BPEM [75] 2018 64.6 66.9 ST-GCN [25] 2018 70.7 73.2 SR-TSL [15] 2018 74.1 79.9 TSRJI [76] 2019 67.9 62.8 2s-AGCN [26] 2019 82.9 84.9 SGN [72] 2020 79.2 81.5 2s-Shift-GCN [29] 2020 85.3 86.6 4s-Shift-GCN [29] 2020 85.9 87.6 MS-G3D [28] 2020 86.9 88.4 Dynamic GCN [30] 2020 87.3 88.6 RA-GCN [49] 2021 81.1 82.7 ST-TR [36] 2021 84.3 86.7 ST-TR-agcn [36] 2021 85.1 87.1 SEFN [31] 2021 86.2 87.8 MSTGNN [73] 2021 87.4 87.6 LAGA [51] 2022 81 82.2 Ta-CNN [24] 2022 85.7 87.3 Graph2Net [32] 2022 86 87.6 CD-GCN [34] 2022 86.3 87.8 FR-AGCN [52] 2022 86.…”

Section: Methodsmentioning

confidence: 99%

“…Year CSv1(%) CSv2(%) ST-GCN [25] 2018 30.25 56.14 DGNN [39] 2019 29.9 N/A 2s-AGCN [26] 2019 34.84 66.68 HARD-Net [77] 2020 36.97 N/A 4s-Shift-GCN [29] 2020 37.98 67.04 AAGCN [27] 2020 41.43 N/A MS-G3D [28] 2020 43.94 N/A PB-GCN [65] 2022 37.48 N/A TE-GCN [68] 2022 42.5 68.2 FR-AGCN [52] 2022…”

Section: Methodsmentioning

confidence: 99%

“…Later, they [27] further proposed a multi-stream network (MS-AAGCN) by incorporating two more streams respectively with the joint motion and bone motion. Most of the recent SOTA GCN-based methods follow the multi-stream network architecture [28], [29], [31], [32], [48]- [52].…”

Section: Gcn-based Action Recognitionmentioning

confidence: 99%

See 3 more Smart Citations

Motion-Driven Spatial and Temporal Adaptive High-Resolution Graph Convolutional Networks for Skeleton-Based Action Recognition

Huang

Dou

Lin

et al. 2023

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

Graph convolutional networks (GCN) have attracted increasing interest in action recognition in recent years. GCN models human skeleton sequences as spatio-temporal graphs. Also, attention mechanisms are often jointly used with GCNs to highlight important frames or body joints in a sequence. However, attention modules learn parameters offline and are fixed, so may not adapt well to various action samples. In this paper, we propose a simple but effective motion-driven spatial and temporal adaptation strategy to dynamically strengthen the features of important frames and joints for skeleton-based action recognition. The rationale is that the joints and frames with dramatic motions are generally more informative and discriminative. We decouple and combine the spatial and temporal refinements by using a two-branch structure, in which the joint and frame-wise feature refinements perform in parallel. Such a structure can also lead to learn more complementary feature representations. Moreover, we propose to use the fully connected graph convolution to learn the long-range spatial dependencies. Besides, we investigate two high-resolution skeleton graphs by creating virtual joints, aiming to improve the representation of skeleton features. By combining the above proposals, we develop a novel motion-driven spatial and temporal adaptive high-resolution GCN. Experimental results demonstrate that the proposed model achieves state-of-the-art (SOTA) results on the challenging large-scale Kinetics-Skeleton and UAV-Human datasets, and it is on par with the SOTA methods on the two NTU-RGB+D 60&120 datasets. Additionally, our motiondriven adaptation method shows encouraging performance when compared with the attention mechanisms.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Gcn-based Action Recognitionmentioning

confidence: 99%

See 2 more Smart Citations

Motion-Driven Spatial and Temporal Adaptive High-Resolution Graph Convolutional Networks for Skeleton-Based Action Recognition

Huang

Dou

Lin

et al. 2023

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

show abstract

“…This model has become the baseline comparison for subsequent studies on action recognition. Follow-up GCN-based work mainly contains adaptive spatial graph kernels or bi-directional skeleton sequences [ 35 , 36 , 37 , 38 ], and

kernel in paper [ 35 ] calculates the similarity of any two joints in each frame and is similar to the attention mechanism. Qin and Liu [ 39 ] introduced angular encoding to fuse higher-order features in GNN at the channel dimension of original inputs.…”

Section: Related Workmentioning

confidence: 99%

Improving Small-Scale Human Action Recognition Performance Using a 3D Heatmap Volume

Yuan

Wang

et al. 2023

Sensors

Self Cite

View full text Add to dashboard Cite

In recent years, skeleton-based human action recognition has garnered significant research attention, with proposed recognition or segmentation methods typically validated on large-scale coarse-grained action datasets. However, there remains a lack of research on the recognition of small-scale fine-grained human actions using deep learning methods, which have greater practical significance. To address this gap, we propose a novel approach based on heatmap-based pseudo videos and a unified, general model applicable to all modality datasets. Leveraging anthropometric kinematics as prior information, we extract common human motion features among datasets through an ad hoc pre-trained model. To overcome joint mismatch issues, we partition the human skeleton into five parts, a simple yet effective technique for information sharing. Our approach is evaluated on two datasets, including the public Nursing Activities and our self-built Tai Chi Action dataset. Results from linear evaluation protocol and fine-tuned evaluation demonstrate that our pre-trained model effectively captures common motion features among human actions and achieves steady and precise accuracy across all training settings, while mitigating network overfitting. Notably, our model outperforms state-of-the-art models in recognition accuracy when fusing joint and limb modality features along the channel dimension.

show abstract

Fusion Attention Graph Convolutional Network with Hyperskeleton for UAV Action Recognition

Liu,

Huang,

Dai

et al. 2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Forward-reverse adaptive graph convolutional networks for skeleton-based action recognition

Cited by 14 publications

References 19 publications

Motion-Driven Spatial and Temporal Adaptive High-Resolution Graph Convolutional Networks for Skeleton-Based Action Recognition

Motion-Driven Spatial and Temporal Adaptive High-Resolution Graph Convolutional Networks for Skeleton-Based Action Recognition

Improving Small-Scale Human Action Recognition Performance Using a 3D Heatmap Volume

Fusion Attention Graph Convolutional Network with Hyperskeleton for UAV Action Recognition

Contact Info

Product

Resources

About