Tensor-based linear dynamical systems for action recognition from 3D skeletons

Ding, Wenwen; Li, Kai; Belyaev, Evgeny; Cheng, Fei

doi:10.1016/j.patcog.2017.12.004

Cited by 39 publications

(25 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Human action recognition approaches can be categorised into two types: methods that are discrete and operate on either image [6] or pre-segmented videos [7,8,9]; and methods that operate over continuous fine-grained action videos [10,11,12]. Even though discrete methods have demonstrated greater performance [13,14], they are disconnected from real world scenarios that are always composed of fine-grained actions. This has been the motivation for researchers to focus on methods that process continuous fine-grained videos.…”

Section: Related Workmentioning

confidence: 99%

Fine-grained action segmentation using the semi-supervised action GAN

Gammulle

Denman

Sridharan

et al. 2020

Pattern Recognition

View full text Add to dashboard Cite

In this paper we address the problem of continuous fine-grained action segmentation, in which multiple actions are present in an unsegmented video stream. The challenge for this task lies in the need to represent the hierarchical nature of the actions and to detect the transitions between actions, allowing us to localise the actions within the video effectively. We propose a novel recurrent semi-supervised Generative Adversarial Network (GAN) model for continuous fine-grained human action segmentation. Temporal context information is captured via a novel Gated Context Extractor (GCE) module, composed of gated attention units, that directs the queued context information through the generator model, for enhanced action segmentation. The GAN is made to learn features in a semi-supervised manner, enabling the model to perform action classification jointly with the standard, unsupervised, GAN learning procedure.We perform extensive evaluations on different architectural variants to demonstrate the importance of the proposed network architecture, and show that it is capable of outperforming current state-of-the-art on three challenging datasets: 50 Salads, MERL Shopping and Georgia Tech Egocentric Activities dataset.

show abstract

Section: Related Workmentioning

confidence: 99%

Fine-grained action segmentation using the semi-supervised action GAN

Gammulle

Denman

Sridharan

et al. 2020

Pattern Recognition

View full text Add to dashboard Cite

show abstract

“…TABLE IV A COMPARISON BETWEEN THE PROPOSED METHOD AND STATE-OF-THE-ART APPROACHES IN TERMS OF NORTH WESTERN UCLA DATASET. Paper Cross-subject Cross-view Virtual view [34] 50.70 47.80 Hankelet [35] 54.20 45.20 MST-AOG [26] 81.60 73.30 Action Bank [36] 24.60 17.60 Poselet [37] 54.90 24.50 Denoised-LSTM [38] -79.57 tLDS [39] 92 It can be seen in Table IV, Virtual view [34] and Hanklet [35] methods are limited in their performance which reflects the challenges of the North Western UCLA dataset (e.g. noise, cluttered backgrounds and various view points).…”

Section: A North Western Ucla Datasetmentioning

confidence: 99%

Multi-view region-adaptive multi-temporal DMM and RGB action recognition

Al-Faris

Chiverton

Yang

et al. 2020

Pattern Anal Applic

View full text Add to dashboard Cite

Human action recognition remains an important yet challenging task. This work proposes a novel action recognition system. It uses a novel Multiple View Region Adaptive Multi-resolution in time Depth Motion Map (MV-RAMDMM) formulation combined with appearance information. Multiple stream 3D Convolutional Neural Networks (CNNs) are trained on the different views and time resolutions of the region adaptive Depth Motion Maps. Multiple views are synthesised to enhance the view invariance. The region adaptive weights, based on localised motion, accentuate and differentiate parts of actions possessing faster motion. Dedicated 3D CNN streams for multi-time resolution appearance information (RGB) are also included. These help to identify and differentiate between small object interactions. A pre-trained 3D-CNN is used here with fine-tuning for each stream along with multiple class Support Vector Machines (SVM)s. Average score fusion is used on the output. The developed approach is capable of recognising both human action and human-object interaction. Three public domain datasets including: MSR 3D Action, Northwestern UCLA multi-view actions and MSR 3D daily activity are used to evaluate the proposed solution. The experimental results demonstrate the robustness of this approach compared with state-of-the-art algorithms.

show abstract

“…When recognizing human actions, the action representations embodying the temporal dynamics can provide a more relevant description than using static data [6]. A linear dynamical system (LDS) [7] is an effective tool in various disciplines for capturing the spatiotemporal data.…”

Section: Introductionmentioning

confidence: 99%

“…A linear dynamical system (LDS) [7] is an effective tool in various disciplines for capturing the spatiotemporal data. Hence, the authors of [6,8] employed an LDS model to capture the spatiotemporal information of an action (skeleton sequence) and used the singular value decomposition (SVD) [9] or Tucker decomposition [10] to estimate model parameters (A, C). The parameters were used to build finite observability matrix O 푇 푘 = [C 푇 , (CA) 푇 , (CA 2 ) 푇 , .…”

Section: Introductionmentioning

confidence: 99%

Nonnegative Tensor‐Based Linear Dynamical Systems for Recognizing Human Action from 3D Skeletons

Cheng

et al. 2019

Mathematical Problems in Engineering

Self Cite

View full text Add to dashboard Cite

Recently, skeleton-based action recognition has become a very important topic in the field of computer vision. It is a challenging task to accurately build a human action model and precisely distinguish similar human actions. In this paper, an action (skeleton sequence) is represented as a third-order nonnegative tensor time series to capture the original spatiotemporal information of the action. As a linear dynamical system (LDS) is an efficient tool for encoding the spatiotemporal data in various disciplines, this paper proposes a nonnegative tensor-based LDS (nLDS) to model the third-order nonnegative tensor time series. Nonnegative Tucker decomposition (NTD) is utilized to estimate the parameters of the nLDS model. These parameters are used to build extended observability sequence O∞T for the action, which implies that O∞T can be considered as the feature descriptor of the action. To avoid the limitations introduced by approximating O∞T with a finite-order matrix, we represent an action as a point on infinite Grassmann manifold comprising the orthonormalized extended observability sequences. The classification task can be performed by dictionary learning and sparse coding on the infinite Grassmann manifold. The experimental results on the MSR-Action3D, UTKinect-Action, and G3D-Gaming datasets demonstrate that the proposed approach achieves a better performance in comparison with the state-of-the-art methods.

show abstract

Tensor-based linear dynamical systems for action recognition from 3D skeletons

Cited by 39 publications

References 13 publications

Fine-grained action segmentation using the semi-supervised action GAN

Fine-grained action segmentation using the semi-supervised action GAN

Multi-view region-adaptive multi-temporal DMM and RGB action recognition

Nonnegative Tensor‐Based Linear Dynamical Systems for Recognizing Human Action from 3D Skeletons

Contact Info

Product

Resources

About