A weighting scheme for mining key skeletal joints for human action recognition

Shabaninia, Elham; Naghsh-Nilchi, Ahmad Reza; Kasaei, Shohreh

doi:10.1007/s11042-019-7740-z

Cited by 6 publications

(4 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In some approaches, the features computed from the action sequences are clustered into posture visual words (representing the prototypical poses of actions), and then the temporal evolutions of those visual words are modeled by explicit methods such as hidden Markov models (HMM) [68,69] or conditional random fields (CRF) [70,71]. Some other approaches consider the manifold of the trajectories [72] or use hierarchical extended histogram (HEH) for modeling temporal variation of features acquired from individual frames of input sequence [73]. These hand-crafted features and descriptors are recently substituted with deep representations to automatically extract high-level information from training data without using hand-crafted rules.…”

Section: Temporal Modeling In Action Recognition Methodsmentioning

confidence: 99%

Transformers in Action Recognition: A Review on Temporal Modeling

Shabaninia¹,

Nezamabadi‐pour²,

Shafizadegan³

2023

Preprint

View full text Add to dashboard Cite

In vision-based action recognition, spatio-temporal features from different modalities are used for recognizing activities. Temporal modeling is a long challenge of action recognition. However, there are limited methods such as pre-computed motion features, three-dimensional (3D) filters, and recurrent neural networks (RNN) for modeling motion information in deep-based approaches. Recently, transformers' success in modeling long-range dependencies in natural language processing (NLP) tasks has gotten great attention from other domains; including speech, image, and video, to rely entirely on self-attention without using sequence-aligned RNNs or convolutions. Although the application of transformers to action recognition is relatively new, the amount of research proposed on this topic within the last few years is astounding. This paper especially reviews recent progress in deep learning methods for modeling temporal variations. It focuses on action recognition methods that use transformers for temporal modeling, discussing their main features, used modalities, and identifying opportunities and challenges for future research.

show abstract

Section: Temporal Modeling In Action Recognition Methodsmentioning

confidence: 99%

Transformers in Action Recognition: A Review on Temporal Modeling

Shabaninia¹,

Nezamabadi‐pour²,

Shafizadegan³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…The average activity accuracy levels obtained were, respectively, 93.5% and 92.3%. In 2019, Shabaninia et al [41] considered weighted 3D joints for human activity presentation. The average accuracy level obtained was 94.94%.…”

Section: Comparisons With the State-of-the-art Methodsmentioning

confidence: 99%

“…Dynamic Bayesian Mixture Model [23] 2014 91.9% Support Vector Machine + Hidden Markov Model [26] 2015 77.3% Multiclass Support Vector Machine [25] 2016 93.5 Classifier Ensemble [12] 2018 92.3% Weighted 3D joints [41] 2019 94.4% Our System 2020 95.5%…”

Section: Year Acc (%)mentioning

confidence: 99%

Prediction of Human Activities Based on a New Structure of Skeleton Features and Deep Learning Model

Jaouedi¹,

Perales

Buades

et al. 2020

Sensors

View full text Add to dashboard Cite

The recognition of human activities is usually considered to be a simple procedure. Problems occur in complex scenes involving high speeds. Activity prediction using Artificial Intelligence (AI) by numerical analysis has attracted the attention of several researchers. Human activities are an important challenge in various fields. There are many great applications in this area, including smart homes, assistive robotics, human–computer interactions, and improvements in protection in several areas such as security, transport, education, and medicine through the control of falling or aiding in medication consumption for elderly people. The advanced enhancement and success of deep learning techniques in various computer vision applications encourage the use of these methods in video processing. The human presentation is an important challenge in the analysis of human behavior through activity. A person in a video sequence can be described by their motion, skeleton, and/or spatial characteristics. In this paper, we present a novel approach to human activity recognition from videos using the Recurrent Neural Network (RNN) for activity classification and the Convolutional Neural Network (CNN) with a new structure of the human skeleton to carry out feature presentation. The aims of this work are to improve the human presentation through the collection of different features and the exploitation of the new RNN structure for activities. The performance of the proposed approach is evaluated by the RGB-D sensor dataset CAD-60. The experimental results show the performance of the proposed approach through the average error rate obtained (4.5%).

show abstract

“…Skeletal data, used as high-level information, is robust against different views, backgrounds, and motion speeds (Shabaninia et al 2019). However, sparse information of 3D joints in skeletal data is insufficient to model human actions, especially human-object interactions fully.…”

Section: Rgb and Skeletonmentioning

confidence: 99%

Multimodal vision-based human action recognition using deep learning: a review

Shafizadegan,

Naghsh-Nilchi,

Shabaninia

2024

Artif Intell Rev

Self Cite

View full text Add to dashboard Cite

Vision-based Human Action Recognition (HAR) is a hot topic in computer vision. Recently, deep-based HAR has shown promising results. HAR using a single data modality is a common approach; however, the fusion of different data sources essentially conveys complementary information and improves the results. This paper comprehensively reviews deep-based HAR methods using multiple visual data modalities. The main contribution of this paper is categorizing existing methods into four levels, which provides an in-depth and comparable analysis of approaches in various aspects. So, at the first level, proposed methods are categorized based on the employed modalities. At the second level, methods categorized in the first level are classified based on the employment of complete modalities or working with missing modalities at the test time. At the third level, complete and missing modality branches are categorized based on existing approaches. Finally, similar frameworks in the third category are grouped together. In addition, a comprehensive comparison is provided for publicly available benchmark datasets, which helps to compare and choose suitable datasets for a task or to develop new datasets. This paper also compares the performance of state-of-the-art methods on benchmark datasets. The review concludes by highlighting several future directions.

show abstract

A weighting scheme for mining key skeletal joints for human action recognition

Cited by 6 publications

References 42 publications

Transformers in Action Recognition: A Review on Temporal Modeling

Transformers in Action Recognition: A Review on Temporal Modeling

Prediction of Human Activities Based on a New Structure of Skeleton Features and Deep Learning Model

Multimodal vision-based human action recognition using deep learning: a review

Contact Info

Product

Resources

About