Action Recognition Algorithm of Spatio–Temporal Differential LSTM Based on Feature Enhancement

Hu, Kai; Zheng, Fei; Weng, Liguo; Ding, Yiwu; Jin, Junlan

doi:10.3390/app11177876

Cited by 14 publications

(8 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The research of RNN (recurrent neural network) began in the 1980s and 1990s and developed into one of the classical deep learning algorithms in the early 21st century. Long short-term Memory Networks (LSTM) are one of the most common recurrent neural networks [178]. LSTM is a variant of RNN, which remembers a controllable amount of previous training data or forgets it more properly [179].…”

Section: Rnn With Vslammentioning

confidence: 99%

An Overview on Visual SLAM: From Tradition to Semantic

Chen

Shang

et al. 2022

Remote Sensing

Self Cite

View full text Add to dashboard Cite

Visual SLAM (VSLAM) has been developing rapidly due to its advantages of low-cost sensors, the easy fusion of other sensors, and richer environmental information. Traditional visionbased SLAM research has made many achievements, but it may fail to achieve wished results in challenging environments. Deep learning has promoted the development of computer vision, and the combination of deep learning and SLAM has attracted more and more attention. Semantic information, as high-level environmental information, can enable robots to better understand the surrounding environment. This paper introduces the development of VSLAM technology from two aspects: traditional VSLAM and semantic VSLAM combined with deep learning. For traditional VSLAM, we summarize the advantages and disadvantages of indirect and direct methods in detail and give some classical VSLAM open-source algorithms. In addition, we focus on the development of semantic VSLAM based on deep learning. Starting with typical neural networks CNN and RNN, we summarize the improvement of neural networks for the VSLAM system in detail. Later, we focus on the help of target detection and semantic segmentation for VSLAM semantic information introduction. We believe that the development of the future intelligent era cannot be without the help of semantic technology. Introducing deep learning into the VSLAM system to provide semantic information can help robots better perceive the surrounding environment and provide people with higher-level help.

show abstract

Section: Rnn With Vslammentioning

confidence: 99%

An Overview on Visual SLAM: From Tradition to Semantic

Chen

Shang

et al. 2022

Remote Sensing

Self Cite

View full text Add to dashboard Cite

show abstract

“…The matrix form is shown in Equation (7). Similarly, the coordination matrix of the remaining two coordinates can be calculated by using Equation (6).…”

Section: Coordination Attention Modulementioning

confidence: 99%

“…With the rapid development of artificial intelligence algorithms, motion-recognition technology, which is an important part of artificial intelligence, is being studied for its application in many fields, such as human–computer interaction, video surveillance, film and television production, and other areas [ 1 , 2 , 3 ]. Many researchers [ 4 , 5 , 6 ] have invested a great deal of energy in this field and designed many excellent algorithms. Among them, most of the traditional algorithms use manual feature extraction, and these algorithms have made a breakthrough [ 7 ].…”

Section: Introductionmentioning

confidence: 99%

Multiple Attention Mechanism Graph Convolution HAR Model Based on Coordination Theory

Ding

Jin

et al. 2022

Sensors

Self Cite

View full text Add to dashboard Cite

Human action recognition (HAR) is the foundation of human behavior comprehension. It is of great significance and can be used in many real-world applications. From the point of view of human kinematics, the coordination of limbs is an important intrinsic factor of motion and contains a great deal of information. In addition, for different movements, the HAR algorithm provides important, multifaceted attention to each joint. Based on the above analysis, this paper proposes a HAR algorithm, which adopts two attention modules that work together to extract the coordination characteristics in the process of motion, and strengthens the attention of the model to the more important joints in the process of moving. Experimental data shows these two modules can improve the recognition accuracy of the model on the public HAR dataset (NTU-RGB + D, Kinetics-Skeleton).

show abstract

“…Shahroudy et al [ 14 ] transformed the 3D coordinates of human joints into a time series, with an RNN leveraged for feature extraction. Echoing the approach of [ 14 ], a multitude of contemporary methods have adopted RNNs and reported promising outcomes [ 15 , 16 , 17 ]. Conversely, a CNN can transform skeleton data into pseudo-images to simulate spatiotemporal dynamics.The dual-stream CNN methodology [ 18 ] introduces a skeleton transformer module for learning joint representation.…”

Section: Introductionmentioning

confidence: 99%

Action Recognition Based on Multi-Level Topological Channel Attention of Human Skeleton

Hu,

Shen,

Wang

et al. 2023

Sensors

Self Cite

View full text Add to dashboard Cite

In action recognition, obtaining skeleton data from human poses is valuable. This process can help eliminate negative effects of environmental noise, including changes in background and lighting conditions. Although GCN can learn unique action features, it fails to fully utilize the prior knowledge of human body structure and the coordination relations between limbs. To address these issues, this paper proposes a Multi-level Topological Channel Attention Network algorithm: Firstly, the Multi-level Topology and Channel Attention Module incorporates prior knowledge of human body structure using a coarse-to-fine approach, effectively extracting action features. Secondly, the Coordination Module utilizes contralateral and ipsilateral coordinated movements in human kinematics. Lastly, the Multi-scale Global Spatio-temporal Attention Module captures spatiotemporal features of different granularities and incorporates a causal convolution block and masked temporal attention to prevent non-causal relationships. This method achieved accuracy rates of 91.9% (Xsub), 96.3% (Xview), 88.5% (Xsub), and 90.3% (Xset) on NTU-RGB+D 60 and NTU-RGB+D 120, respectively.

show abstract

Action Recognition Algorithm of Spatio–Temporal Differential LSTM Based on Feature Enhancement

Cited by 14 publications

References 23 publications

An Overview on Visual SLAM: From Tradition to Semantic

An Overview on Visual SLAM: From Tradition to Semantic

Multiple Attention Mechanism Graph Convolution HAR Model Based on Coordination Theory

Action Recognition Based on Multi-Level Topological Channel Attention of Human Skeleton

Contact Info

Product

Resources

About