Human Behavior Analysis: A Survey on Action Recognition

Degardin, Bruno; Proença, Hugo

doi:10.3390/app11188324

Cited by 15 publications

(8 citation statements)

References 79 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, many modern trajectory datasets are increasingly providing action information. Action prediction tasks from video data achieve high performances [7], and hence can be performed efficiently and reliably as a pre-processing step for our trajectory prediction framework. We deem evaluating such idea as future work.…”

Section: Discussionmentioning

confidence: 99%

Action-based Contrastive Learning for Trajectory Prediction

Halawa¹,

Hellwich²,

Bideau³

2022

Preprint

View full text Add to dashboard Cite

Trajectory prediction is an essential task for successful humanrobot interaction, such as in autonomous driving. In this work, we address the problem of predicting future pedestrian trajectories in a firstperson view setting with a moving camera. To that end, we propose a novel action-based contrastive learning loss, that utilizes pedestrian action information to improve the learned trajectory embeddings. The fundamental idea behind this new loss is that trajectories of pedestrians performing the same action should be closer to each other in the feature space than the trajectories of pedestrians with significantly different actions. In other words, we argue that behavioral information about pedestrian action influences their future trajectory. Furthermore, we introduce a novel sampling strategy for trajectories that is able to effectively increase negative and positive contrastive samples. Additional synthetic trajectory samples are generated using a trained Conditional Variational Autoencoder (CVAE), which is at the core of several models developed for trajectory prediction. Results show that our proposed contrastive framework employs contextual information about pedestrian behavior, i.e. action, effectively, and it learns a better trajectory representation. Thus, integrating the proposed contrastive framework within a trajectory prediction model improves its results and outperforms state-of-the-art methods on three trajectory prediction benchmarks [31,32,26].

show abstract

Section: Discussionmentioning

confidence: 99%

Action-based Contrastive Learning for Trajectory Prediction

Halawa¹,

Hellwich²,

Bideau³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…A CNN can well process two-dimensional grid data such as images [16][17][18][19][20], but most of people's daily life involves non-Euclideanspace data, so more and more people are engaged in research on GCNs [21][22][23][24][25][26]. There are two main methods used for GCNs: the spatial-based method and the spectral-based method.…”

Section: Graph Convolutional Networkmentioning

confidence: 99%

Skeleton Motion Recognition Based on Multi-Scale Deep Spatio-Temporal Features

Ding

Jin

et al. 2022

Applied Sciences

View full text Add to dashboard Cite

In the task of human motion recognition, the overall action span is changeable, and there may be an inclusion relationship between action semantics. This paper proposes a novel multi-scale time sampling module and a deep spatiotemporal feature extraction module, which strengthens the receptive field of the feature map and strengthens the extraction of spatiotemporal-related feature information via the network. We study and compare the performance of three existing multi-channel fusion methods to improve the recognition accuracy of the network on the open skeleton recognition dataset. In this paper, several groups of comparative experiments are carried out on two public datasets. The experimental results show that compared with the classical 2s-AGCN algorithm, the accuracy of the algorithm proposed in this paper shows an improvement of 1% on the Kinetics dataset and 0.4% and 1% on the two evaluating indicators of the NTU-RGB+D dataset, respectively.

show abstract

“…Action recognition is core work in video understanding [ 12 ], which refers to recognizing an action in a video based on the complete execution of the action, reconciling the characteristics of video image data to achieve high-level understanding. It has been studied for decades, solving many problems such as abnormal event detection [ 13 ], video retrieval [ 14 ], intelligent robots [ 15 ], and visual surveillance [ 16 ].…”

Section: Introductionmentioning

confidence: 99%

Convolutional Neural Networks or Vision Transformers: Who Will Win the Race for Action Recognitions in Visual Data?

Moutik

Sekkat

Tigani

et al. 2023

Sensors

View full text Add to dashboard Cite

Understanding actions in videos remains a significant challenge in computer vision, which has been the subject of several pieces of research in the last decades. Convolutional neural networks (CNN) are a significant component of this topic and play a crucial role in the renown of Deep Learning. Inspired by the human vision system, CNN has been applied to visual data exploitation and has solved various challenges in various computer vision tasks and video/image analysis, including action recognition (AR). However, not long ago, along with the achievement of the transformer in natural language processing (NLP), it began to set new trends in vision tasks, which has created a discussion around whether the Vision Transformer models (ViT) will replace CNN in action recognition in video clips. This paper conducts this trending topic in detail, the study of CNN and Transformer for Action Recognition separately and a comparative study of the accuracy-complexity trade-off. Finally, based on the performance analysis’s outcome, the question of whether CNN or Vision Transformers will win the race will be discussed.

show abstract

Human Behavior Analysis: A Survey on Action Recognition

Cited by 15 publications

References 79 publications

Action-based Contrastive Learning for Trajectory Prediction

Action-based Contrastive Learning for Trajectory Prediction

Skeleton Motion Recognition Based on Multi-Scale Deep Spatio-Temporal Features

Convolutional Neural Networks or Vision Transformers: Who Will Win the Race for Action Recognitions in Visual Data?

Contact Info

Product

Resources

About