Action Recognition Using Attention-Joints Graph Convolutional Neural Networks

Abstract

Human skeleton contains significant information about actions, therefore, it is quite intuitive to incorporate skeletons in human action recognition. Human skeleton resembles to a graph where body joints and bones mimic to graph nodes and edges. This resemblance of human skeleton to graph structure is the main motivation to apply graph convolutional neural network for human action recognition. Results show that the discriminant contribution of different joints is not equal for different actions. Therefore, we propose to use attention-joints that correspond to joints significantly contributing to the specific actions. Features corresponding to only these attention-joints are computed and assigned as node features of the graph. In our method, node features (also termed as attention-joint features) include the i) distances of attention-joints from the center-of-gravity of human body, ii) distances between adjacent attention-joints and iii) joints flow features. The proposed method gives a simple but more efficient representation of skeleton sequences by concatenating more relative distances and relative coordinates to other joints. The proposed methodology has been evaluated on single image Stanford 40-Actions dataset, as well as on temporal skeleton-based action recognition PKU-MDD and NTU-RGBD datasets. Results show that this framework outperforms existing state-of-the-art methods. INDEX TERMS Human action recognition, attention-joints, graph convolutional neural network.

View-Adaptive Graph Neural Network for Action Recognition

“…The spatiotemporal graph convolution neural network is introduced by treating human joints as nodes and bones as edges of a graph. Several recent works [2], [6], [34], [35], [32], [20], [36], [37], [38], [39], attempt to convert 3D skeleton to graph and then leverage graph convolution neural networks which significantly outperformed previous approaches.…”

Open Set Domain Recognition via Attention-Based GCN and Semantic Matching Optimization

“…Attention-Based GCN. Attention mechanism has proven to be useful in many tasks (Liu et al [23]; Ahmad et al [24]). One of the benefits of attention mechanism is that it emphasises on the most relevant parts of the input.…”

Higher-Order Graph Convolutional Networks With Multi-Scale Neighborhood Pooling for Semi-Supervised Node Classification

“…This is mainly due to the increased quantity of graph-structured data in real-world applications and the weak learning ability of convolutional neural networks (CNNs) when working with these data. GCNs and their variants have achieved promising performance with respect to Euclidean and non-Euclidean data, including but not limited to computer vision [1], [2], natural language processing [3], [4], publication citations [5], [6], social relationships [7], traffic prediction [12], [13], point clouds [8], [9], action recognition [10], [11], and recommender systems [14]- [16]. On the one hand, there is a large quantity of unlabeled graph data in nature, and labeling these data is very expensive.…”