Learning Graph Convolutional Network for Skeleton-Based Human Action Recognition by Neural Searching

Abstract

Human action recognition from skeleton data, fuelled by the Graph Convolutional Network (GCN) with its powerful capability of modeling non-Euclidean data, has attracted lots of attention. However, many existing GCNs provide a pre-defined graph structure and share it through the entire network, which can loss implicit joint correlations especially for the higher-level features. Besides, the mainstream spectral GCN is approximated by one-order hop such that higher-order connections are not well involved. All of these require huge efforts to design a better GCN architecture. To address these problems, we turn to Neural Architecture Search (NAS) and propose the first automatically designed GCN for this task. Specifically, we explore the spatial-temporal correlations between nodes and build a search space with multiple dynamic graph modules. Besides, we introduce multiple-hop modules and expect to break the limitation of representational capacity caused by one-order approximation. Moreover, a corresponding sampling- and memory-efficient evolution strategy is proposed to search in this space. The resulted architecture proves the effectiveness of the higher-order approximation and the layer-wise dynamic graph modules. To evaluate the performance of the searched model, we conduct extensive experiments on two very large scale skeleton-based action recognition datasets. The results show that our model gets the state-of-the-art results in term of given metrics.

Exploring Deep Learning-Based Architecture, Strategies, Applications and Current Trends in Generic Object Detection: A Comprehensive Review

“…Due to the success of representation learning trends in many areas, graph representation learning has recently emerged in deep learning models and has performed well in a variety of issues. Graph convolutional networks are one of the most prominent graph-deep learning models [136] such as spatial-temporal graph convolutional model [137], the skeleton-based method [138] and GCN used to process the image scene [139]. To better understand Graph Conventional Network, you can refer to this survey [140].…”

Global Co-Occurrence Feature and Local Spatial Feature Learning for Skeleton-Based Action Recognition

“…In the Kinetics dataset, we compare our model with eight state-of-the-art approaches. These eight approaches can be divided into four categories: traditional method [ 44 ], LSTM-based method [ 17 ], CNN-based method [ 24 ], and GCN-based methods [ 4 , 6 , 7 , 41 , 42 ]. Table 2 presents the top-1 and top-5 classification performances.…”

Enhanced Spatial and Extended Temporal Graph Convolutional Network for Skeleton-Based Action Recognition

“…Ref. [ 33 ] changes this kind of operation and proposes an automatically designed GCN by neural architecture searching. Multiple dynamic graph modules are provided to the search space and the optimal one is chosen for each layer.…”

“…Based on this judgment, Yan et al [ 23 ] proposed a spatial-temporal graph convolutional network (ST-GCN) representing human joints as vertices and the bones as edges. The ST-GCN improves the accuracy of action recognition to a new level, and substantial ST-GCNs are subsequently proposed based on it [ 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 ]. However, there are still two problems to be addressed in these methods.…”