Learning features combination for human action recognition from skeleton sequences

Luvizon, Diogo C.; Tabia, Hedi; Picard, David

doi:10.1016/j.patrec.2017.02.001

Cited by 83 publications

(20 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The recognition rate of our approach on the UCKinect Dataset is 98.87%. Our method outperforms the Lie group [5], Grassmann manifold [16], Eigenjoints [29], and learning feature combination [30], which achieved recognition rates of 97.08, 97.91,97.1, and 98.00%, respectively, as shown in Table 4.…”

Section: Experiments On the Uckinect-action Datasetmentioning

confidence: 87%

Key-Skeleton-Pattern Mining on 3D Skeletons Represented by Lie Group for Action Recognition

Cheng

et al. 2018

Mathematical Problems in Engineering

View full text Add to dashboard Cite

The human skeleton can be considered as a tree system of rigid bodies connected by bone joints. In recent researches, substantial progress has been made in both theories and experiments on skeleton-based action recognition. However, it is challenging to accurately represent the skeleton and precisely eliminate noisy skeletons from the action sequence. This paper proposes a novel skeletal representation, which is composed of two subfeatures to recognize human action: static features and dynamic features. First, to avoid scale variations from subject to subject, the orientations of the rigid bodies in a skeleton are employed to capture the scale-invariant spatial information of the skeleton. The static feature of the skeleton is defined as a combination of the orientations. Unlike previous orientation-based representations, the orientation of a rigid body in the skeleton is defined as the rotations between the rigid body and the coordinate axes in three-dimensional space. Each rotation is mapped to the special orthogonal group SO(3). Next, the rigid-body motions between the skeleton and its previous skeletons are utilized to capture the temporal information of the skeleton. The dynamic feature of the skeleton is defined as a combination of the motions. Similarly, the motions are represented as points in the special Euclidean group SE(3). Therefore, the proposed skeleton representation lies in the Lie group (SE(3)×⋯×SE(3), SO(3)×⋯×SO(3)), which is a manifold. Using the proposed representation, an action can be considered as a series of points in this Lie group. Then, to recognize human action more accurately, a new pattern-growth algorithm named MinP-PrefixSpan is proposed to mine the key-skeleton-patterns from the training dataset. Because the algorithm reduces the number of new patterns in each growth step, it is more efficient than the PrefixSpan algorithm. Finally, the key-skeleton-patterns are used to discover the most informative skeleton sequences of each action (skeleton sequence). Our approach achieves accuracies of 94.70%, 98.87%, and 95.01% on three action datasets, outperforming other relative action recognition approaches, including LieNet, Lie group, Grassmann manifold, and Graph-based model.

show abstract

Section: Experiments On the Uckinect-action Datasetmentioning

confidence: 87%

Key-Skeleton-Pattern Mining on 3D Skeletons Represented by Lie Group for Action Recognition

Cheng

et al. 2018

Mathematical Problems in Engineering

View full text Add to dashboard Cite

show abstract

“…The recognition rate of our approach on the dataset is 98.23%. It is obvious that our approach outperforms SE3 [12], EigenJoints [13], Grassmann manifold [8], Key-Pose-Motifs [18], learning features combination [16], Ensemble TS-LSTM [14], tLDS [6], and Bi-LSTM [15], which achieve recognition rates of 97.08%, 97.10%, 88.5%, 93.47%, 98.00%, 96.97%, 96.48%, and 96.89%, respectively.…”

Section: Approachmentioning

confidence: 93%

“…Recognition Accuracy SE3 [12] 89.48 Grassmann Manifold [8] 91.21 Learning features combination [16] 90.36 ± 2.45 ST-LSTM + Trust Gate [17] 94.80 Bi-LSTM [15] 86.18 Our approach 96.97 approach outperforms various other methods extracting the action feature from 3D joint positions. Our approach achieves an average accuracy of 97.63% for the MSR-Action3D dataset, outperforming the other action recognition approaches.…”

Section: Approachmentioning

confidence: 98%

“…Recognition Accuracy SE3 [12] 97.08 EigenJoints [13] 97.10 Grassmann Manifold [8] 88.5 Key-Pose-Motifs [18] 93.47 Learning features combination [16] 98.00 Ensemble TS-LSTM [14] 96.97 tLDS [6] 96.48 Bi-LSTM [15] 96.89 Our approach 98. 23 10 actions are walking, sitting down, standing up, picking up, carrying, throwing, pushing, pulling, waving, and clapping hands.…”

Section: Approachmentioning

confidence: 99%

See 1 more Smart Citation

Nonnegative Tensor‐Based Linear Dynamical Systems for Recognizing Human Action from 3D Skeletons

Cheng

et al. 2019

Mathematical Problems in Engineering

View full text Add to dashboard Cite

Recently, skeleton-based action recognition has become a very important topic in the field of computer vision. It is a challenging task to accurately build a human action model and precisely distinguish similar human actions. In this paper, an action (skeleton sequence) is represented as a third-order nonnegative tensor time series to capture the original spatiotemporal information of the action. As a linear dynamical system (LDS) is an efficient tool for encoding the spatiotemporal data in various disciplines, this paper proposes a nonnegative tensor-based LDS (nLDS) to model the third-order nonnegative tensor time series. Nonnegative Tucker decomposition (NTD) is utilized to estimate the parameters of the nLDS model. These parameters are used to build extended observability sequence O∞T for the action, which implies that O∞T can be considered as the feature descriptor of the action. To avoid the limitations introduced by approximating O∞T with a finite-order matrix, we represent an action as a point on infinite Grassmann manifold comprising the orthonormalized extended observability sequences. The classification task can be performed by dictionary learning and sparse coding on the infinite Grassmann manifold. The experimental results on the MSR-Action3D, UTKinect-Action, and G3D-Gaming datasets demonstrate that the proposed approach achieves a better performance in comparison with the state-of-the-art methods.

show abstract

“…Accuracy Multi-part Bag-of-Poses [22] 82.00% Riemannian Manifold [23] 87.04% Latent Variables [24] 89.67% Lie Group [9] 90.88% Feature Combinations [25] 94…”

Section: Methodsmentioning

confidence: 99%

3D Human Action Recognition with Siamese-LSTM Based Deep Metric Learning

Akgül¹

2018

JOIG

View full text Add to dashboard Cite

This paper proposes a new 3D Human Action Recognition system as a two-phase system: (1) Deep Metric Learning Module which learns a similarity metric between two 3D joint sequences using Siamese-LSTM networks; (2) A Multiclass Classification Module that uses the output of the first module to produce the final recognition output. This model has several advantages: the first module is trained with a larger set of data because it uses many combinations of sequence pairs. Our deep metric learning module can also be trained independently of the datasets, which makes our system modular and generalizable. We tested the proposed system on standard and newly introduced datasets that showed us that initial results are promising. We will continue developing this system by adding more sophisticated LSTM blocks and by cross-training between different datasets.

show abstract

Learning features combination for human action recognition from skeleton sequences

Cited by 83 publications

References 29 publications

Key-Skeleton-Pattern Mining on 3D Skeletons Represented by Lie Group for Action Recognition

Key-Skeleton-Pattern Mining on 3D Skeletons Represented by Lie Group for Action Recognition

Nonnegative Tensor‐Based Linear Dynamical Systems for Recognizing Human Action from 3D Skeletons

3D Human Action Recognition with Siamese-LSTM Based Deep Metric Learning

Contact Info

Product

Resources

About