Discriminative Multi-instance Multitask Learning for 3D Action Recognition

Yang, Yanhua; Deng, Cheng; Gao, Shangqian; Liu, Wei; Tao, Dapeng; Gao, Xinbo

doi:10.1109/tmm.2016.2626959

Cited by 105 publications

(34 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Lie Group [14] 90.9 2014 LARP+mfPCA [66] 89.7 2015 Rolling Rotations [67] 91.4 2016 SPGK [13] 91.6 2016 Transion Forests [68] 94.2 2017 MIMTL [69] 95.3 2017 Bi-LSTM [22] 93.0 2018 Deep STGC K [33] 99.1 2018 GR-GCN (Bone only) 95.5 GR-GCN (Bone + Intra-connection) 95.6 Complete GR-GCN model 98.4 effectiveness of the proposed graph construction, in which the temporal connectivities are vital.…”

Section: Methods Accuracy Yearmentioning

confidence: 99%

Optimized Skeleton-based Action Recognition via Sparsified Graph Regression

Gao

Tang

et al. 2019

Proceedings of the 27th ACM International Conference on Multimedia

163

105

View full text Add to dashboard Cite

With the prevalence of accessible depth sensors, dynamic human body skeletons have attracted much attention as a robust modality for action recognition. Previous methods model skeletons based on RNN or CNN, which has limited expressive power for irregular skeleton joints. While graph convolutional networks (GCN) have been proposed to address irregular graphstructured data, the fundamental graph construction remains challenging. In this paper, we represent skeletons naturally on graphs, and propose a graph regression based GCN (GR-GCN) for skeleton-based action recognition, aiming to capture the spatio-temporal variation in the data. As the graph representation is crucial to graph convolution, we first propose graph regression to statistically learn the underlying graph from multiple observations. In particular, we provide spatio-temporal modeling of skeletons and pose an optimization problem on the graph structure over consecutive frames, which enforces the sparsity of the underlying graph for efficient representation. The optimized graph not only connects each joint to its neighboring joints in the same frame strongly or weakly, but also links with relevant joints in the previous and subsequent frames. We then feed the optimized graph into the GCN along with the coordinates of the skeleton sequence for feature learning, where we deploy high-order and fast Chebyshev approximation of spectral graph convolution. Further, we provide analysis of the variation characterization by the Chebyshev approximation. Experimental results validate the effectiveness of the proposed graph regression and show that the proposed GR-GCN achieves the state-of-the-art performance on the widely used NTU RGB+D, UT-Kinect and SYSU 3D datasets.

show abstract

Section: Methods Accuracy Yearmentioning

confidence: 99%

Optimized Skeleton-based Action Recognition via Sparsified Graph Regression

Gao

Tang

et al. 2019

Proceedings of the 27th ACM International Conference on Multimedia

163

105

View full text Add to dashboard Cite

show abstract

“…Accuracy Multi-part Bag-of-Poses [18] 82.00% Riemannian Manifold [6] 87.04% Lie Group [10] 90.88% Graph-Based [13] 91.63% MIMTL [59] 95.29% P-LSTM [20] 95.35% A 2 GNN 98.60%…”

Section: Methodsmentioning

confidence: 99%

Action-Attending Graphic Neural Network

Cui

Zheng

et al. 2018

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

The motion analysis of human skeletons is crucial for human action recognition, which is one of the most active topics in computer vision. In this paper, we propose a fully end-to-end action-attending graphic neural network (A2GNN) for skeleton-based action recognition, in which each irregular skeleton is structured as an undirected attribute graph. To extract high-level semantic representation from skeletons, we perform the local spectral graph filtering on the constructed attribute graphs like the standard image convolution operation. Considering not all joints are informative for action analysis, we design an actionattending layer to detect those salient action units (AUs) by adaptively weighting skeletal joints. Herein the filtering responses are parameterized into a weighting function irrelevant to the order of input nodes. To further encode continuous motion variations, the deep features learnt from skeletal graphs are gathered along consecutive temporal slices and then fed into a recurrent gated network. Finally, the spectral graph filtering, action-attending and recurrent temporal encoding are integrated together to jointly train for the sake of robust action recognition as well as the intelligibility of human actions. To evaluate our A2GNN, we conduct extensive experiments on four benchmark skeletonbased action datasets, including the large-scale challenging NTU RGB+D dataset. The experimental results demonstrate that our network achieves the state-of-the-art performances.

show abstract

“…The descriptor compactly characterizes the salient information of a specific action, on which texture features are calculated to represent the action. Yang et al [11] propose a discriminative multiinstance multitask learning framework for human action recognition. For this purpose, they discover the intrinsic relationship between joint configurations and action classes.…”

Section: Related Workmentioning

confidence: 99%

Human Action Recognition in Videos Using Stable Features

Ullah¹,

Ullah²,

Alseadonn³

2017

SIPIJ

View full text Add to dashboard Cite

Human action recognition is still a challenging problem and researchers are focusing to investigate this problem using different techniques. We propose a robust approach for human action recognition. This is achieved by extracting stable spatio-temporal features in terms of pairwise local binary pattern (P-LBP) and scale invariant feature transform (SIFT). These features are used to train an MLP neural network during the training stage, and the action classes are inferred from the test videos during the testing stage. The proposed features well match the motion of individuals and their consistency, and accuracy is higher using a challenging dataset. The experimental evaluation is conducted on a benchmark dataset commonly used for human action recognition. In addition, we show that our approach outperforms individual features i.e. considering only spatial and only temporal feature.

show abstract

Discriminative Multi-instance Multitask Learning for 3D Action Recognition

Cited by 105 publications

References 45 publications

Optimized Skeleton-based Action Recognition via Sparsified Graph Regression

Optimized Skeleton-based Action Recognition via Sparsified Graph Regression

Action-Attending Graphic Neural Network

Human Action Recognition in Videos Using Stable Features

Contact Info

Product

Resources

About