Saliency-based foreground trajectory extraction using multiscale hybrid masks for action recognition

Zhang, Guoliang; Jia, Songmin; Zhang, Xiangyin; Li, Xiuzhi

doi:10.1117/1.jei.27.5.053049

Cited by 3 publications

(2 citation statements)

References 49 publications

(97 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Gupta et al [16] utilized the body pose as a clue for action recognition. Zhang et al [17] proposed a foreground trajectory extraction approach based on a saliency sampling strategy intending to lessen the reduction of valid trajectories of action. Felzenszwalb et al [18] proposed a structural part-based model to represent human actions.…”

Section: Related Workmentioning

confidence: 99%

Deep Ensemble Learning for Human Action Recognition in Still Images

Zhang

et al. 2020

Complexity

View full text Add to dashboard Cite

Numerous human actions such as “Phoning,” “PlayingGuitar,” and “RidingHorse” can be inferred by static cue-based approaches even if their motions in video are available considering one single still image may already sufficiently explain a particular action. In this research, we investigate human action recognition in still images and utilize deep ensemble learning to automatically decompose the body pose and perceive its background information. Firstly, we construct an end-to-end NCNN-based model by attaching the nonsequential convolutional neural network (NCNN) module to the top of the pretrained model. The nonsequential network topology of NCNN can separately learn the spatial- and channel-wise features with parallel branches, which helps improve the model performance. Subsequently, in order to further exploit the advantage of the nonsequential topology, we propose an end-to-end deep ensemble learning based on the weight optimization (DELWO) model. It contributes to fusing the deep information derived from multiple models automatically from the data. Finally, we design the deep ensemble learning based on voting strategy (DELVS) model to pool together multiple deep models with weighted coefficients to obtain a better prediction. More importantly, the model complexity can be reduced by lessening the number of trainable parameters, thereby effectively mitigating overfitting issues of the model in small datasets to some extent. We conduct experiments in Li’s action dataset, uncropped and 1.5x cropped Willow action datasets, and the results have validated the effectiveness and robustness of our proposed models in terms of mitigating overfitting issues in small datasets. Finally, we open source our code for the model in GitHub (https://github.com/yxchspring/deep_ensemble_learning) in order to share our model with the community.

show abstract

Section: Related Workmentioning

confidence: 99%

Deep Ensemble Learning for Human Action Recognition in Still Images

Zhang

et al. 2020

Complexity

View full text Add to dashboard Cite

show abstract

“…Vehicle trajectory data analysis is a research hotspot in intelligent transportation and smart city [ 1 ]. People can deeply understand their life trajectory, social behavior, environmental change, and urban evolution using video GIS, video object recognition, and visual analysis of video object trajectory [ 2 , 3 ]. In recent years, the urban video monitoring system has gradually developed from single-camera processing to multi-camera equipment joint analysis.…”

Section: Introductionmentioning

confidence: 99%

Hierarchical Clustering Algorithm for Multi-Camera Vehicle Trajectories Based on Spatio-Temporal Grouping under Intelligent Transportation and Smart City

Wang

Xie

Tang

2023

Sensors

View full text Add to dashboard Cite

With the emergence of intelligent transportation and smart city system, the issue of how to perform an efficient and reasonable clustering analysis of the mass vehicle trajectories on multi-camera monitoring videos through computer vision has become a significant area of research. The traditional trajectory clustering algorithm does not consider camera position and field of view and neglects the hierarchical relation of the video object motion between the camera and the scenario, leading to poor multi-camera video object trajectory clustering. To address this challenge, this paper proposed a hierarchical clustering algorithm for multi-camera vehicle trajectories based on spatio-temporal grouping. First, we supervised clustered vehicle trajectories in the camera group according to the optimal point correspondence rule for unequal-length trajectories. Then, we extracted the starting and ending points of the video object under each group, hierarchized the trajectory according to the number of cross-camera groups, and supervised clustered the subsegment sets of different hierarchies. This method takes into account the spatial relationship between the camera and video scenario, which is not considered by traditional algorithms. The effectiveness of this approach has been proved through experiments comparing silhouette coefficient and CPU time.

show abstract

Detection Algorithm of Tennis Serve Mistakes Based on Feature Point Trajectory

Tang

2022

Advances in Meteorology

View full text Add to dashboard Cite

To address the issue of high recognition error in conventional action error detection methods, this article proposes a game of tennis serve error action detection algorithm based on feature point trajectory. To begin, a feature detection model for tennis serve images is established, followed by segmentation of the tennis serve images’ multiscale features. Second, the path of the tennis serving image is effectively corrected, thereby raising the bar for tennis training and competition. Additionally, a visual feature acquisition system for tennis serving action is being developed using remote video monitoring in order to correct the path of the serving image during play. The corner mark of the serving action error point is determined using this algorithm, and the optimal modeling of the tennis serving image’s path correction is realized using the developed edge segmentation algorithm. The results of simulations demonstrate that the aforementioned algorithm improves real-time performance and accuracy, and that it can accurately track players’ visual edge information feature points while they are serving, conduct real-time evaluation and guidance via an expert system, effectively correct the tennis serving image path, and enhance your capacity for service.

show abstract

Saliency-based foreground trajectory extraction using multiscale hybrid masks for action recognition

Cited by 3 publications

References 49 publications

Deep Ensemble Learning for Human Action Recognition in Still Images

Deep Ensemble Learning for Human Action Recognition in Still Images

Hierarchical Clustering Algorithm for Multi-Camera Vehicle Trajectories Based on Spatio-Temporal Grouping under Intelligent Transportation and Smart City

Detection Algorithm of Tennis Serve Mistakes Based on Feature Point Trajectory

Contact Info

Product

Resources

About