Human Action Recognition in Unconstrained Trimmed Videos Using Residual Attention Network and Joints Path Signature

Abstract

Action recognition has been achieved great progress in recent years because of better feature representation learning and classification technology like convolutional neural networks (CNNs). However, most current deep learning approaches treat the action recognition as a black box, ignoring the specific domain knowledge of action itself. In this paper, by analyzing the characteristics of different actions, we proposed a new framework that involves residual-attention module and joint path-signature feature (JPSF) representation framework. The path signature theory was developed recently in the field of rough path and stochastic analysis, which provides a very efficient way to analyze any temporal sequence data. The proposed n-fold joint path signature features entail the Euclidean distances between joints and respective angles. For our experiment, JPSF for three modalities of joints (spatial location, bi-folds and tri-folds) are computed over the temporal length of action sequence. Then all these PSF are concatenated and fed to a CNN to give the recognition result. Experiments on three benchmark datasets, J-HMDB, HMDB-51 and UCF-101, indicate that our proposed method achieves state-of-the-art performance. INDEX TERMS Convolutional neural networks, residual-attention, path signature features.

Detecting Human Actions in Drone Images Using YoloV5 and Stochastic Gradient Boosting

“…Over the last decade, recognizing human actions in videos has been challenging and an active area of research among the computer vision community [26][27][28][29][30][31][32][33][34]. However, the recognition and detection of actions in aerial images is a less developed area, and differs from previous work that simply adopts the perspective of pedestrians in the scene [31,35].…”

A composite framework of deep multiple view human joints feature extraction and selection strategy with hybrid adaptive sunflower optimization‐whale optimization algorithm for human action recognition in video sequences

“…Most of the action dataset depends on the path‐signature feature. Thus, Ahmad et al 18 proposed a residual‐attention model and joint path‐signature feature model after investigating the behavior of dissimilar activities in the dataset. Here, each frame's significant features have been emphasized by the attention model and the residual model has been constructed by the usual residual net used in the convolutional network.…”

“…Here, MaxIter is the maximum iterations permitted and "t" is an iteration number. A spiral-shaped path can be simulated, and to form the position of neighbor search agent, a spiral equation is used in Equation (18).…”