Cross-Channel Graph Convolutional Networks for Skeleton-Based Action Recognition

Abstract

In recent years, skeleton-based action recognition, graph convolutional networks, have achieved remarkable performance. In these existing works, the features of all nodes in the neighbor set are aggregated into the updated features of the root node, while these features are located in the same feature channel determined by the same 1 × 1 convolution filter. This may not be optimal for capturing the features of spatial dimensions among adjacent vertices effectively. Besides, the effect of feature channels that are independent of the current action on the performance of the model is rarely investigated in existing methods. In this paper, we propose cross-channel graph convolutional networks for skeleton-based action recognition. The features fusion mechanism in our network is cross-channel, i.e, the updated feature of the root node is derived from different feature channels. Because different feature channels come from different 1 × 1 convolution filters, the cross-channel fusion mechanism significantly improves the ability of the model to capture local features among adjacent vertices. Moreover, by introducing a channel attention mechanism to our model, we suppress the influence of feature channels unrelated to action recognition on model performance, which improves the robustness of the model against the feature channels independent of the current action. Extensive experiments on the two large-scale datasets, NTU-RGB+D and Kinetics-Skeleton, demonstrate that the performance of our model exceeds the current mainstream methods.

Action Recognition for Multiview Skeleton 3D Data Using NTURGB + D Dataset

“…That comparison shows that the proposed method can prefer for action recognition. Multidimensional indexing [25] 84.6 HMM [26] 88.3 PCA and HMM [27] 87.5 Memory-based attention control system [28] 8 0 Dynamic time warping [29] 80.05 Bipartite graph [30] 82.8 Multi-sensor fusion [31] 8 8 Deep learning-based hierarchical feature model [32] 70.32 Deep convolutional neural network [33] 41.5 Collaborative sparse coding [34] 79.18 Convex multiview semi-supervised classification [35] 59.08 Scene flow to action map and ConvNets [36] 61.94 Convolutional neural network [37] 66.29 Multiview fusion [14] 85.9 Graph convolutional networks [38] 88.2 Proposed method 88.89 Fig. 9 shows the confusion matrix for 60 classes where each row instance depicts actual classes and each column as predicted classes.…”

Skeleton Graph-Neural-Network-Based Human Action Recognition: A Survey

“…Methods, such as [ 49 , 73 , 74 , 100 , 120 , 126 , 127 , 131 ], assumed that features in different channels have various importance, and thus they attempted to balance the importance of each channel while inferring, known as channel-wise attention.…”

Fast Temporal Graph Convolutional Model for Skeleton-Based Action Recognition

“…Xie et al [ 6 ] proposed a new network design described as a cross-channel graph convolutional network. This significantly improves the ability of the model to capture local features among adjacent vertices.…”