Spatio-Temporal Transformer for Online Video Understanding

Du, Zexu; Zhang, Guoliang; Lü, Wei; Zhao, Ting; Wu, Peng

doi:10.1088/1742-6596/2171/1/012020

Cited by 2 publications

(2 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Because cross-validation is necessary to obtain reliable and more accurate results. In each cycle, test and training data are randomly divided and this is done 5 times [20].…”

Section: Resultsmentioning

confidence: 99%

Comparison of Video Classification Results with Machine Learning

Akarsu,

Karacali

2024

View full text Add to dashboard Cite

The article is about the classification of anomaly detection dataset-UCF videos obtained from Kaggle. The data set consists of 4-class video files. While three of them consist of different crimes and abnormal events such as Burglary, Explosion, and Fighting, one class consists of videos containing quotes from daily life that do not contain different crimes or events. 4-class videos from Anomaly-Detection-Dataset-UCF were used, Burglary consists of 100 videos, and videos from the other 3 classes are 50 each. and the number of frames of each video varies. First, each video in the dataset was converted into image frames. Because video data is the combination of multiple image frames to form a series. and each picture frame in this series contains important information. To obtain this information, feature information of each frame was extracted. This feature extraction process was carried out with 3 different algorithms. These algorithms; Alexnet is Vgg19 and Resnet18. However, during the feature acquisition phase, the last three layers from each of the 3 pre-training algorithms were removed with Transfer Learning, and 4 layers were manually added instead. and retrained Alexnet, vgg19, and resnet18 pretraining algorithms. Thus, it was aimed to obtain 1500 features instead of 1000 from each pre-training algorithm. It was aimed to obtain more detailed information about the videos by extracting more features. that is, it was aimed to increase the number of features with transfer learning. In other words, in this study, transfer learning was used for feature improvement and the effects of these three different algorithms on classification accuracy with the same parameters were investigated. The classification was done in MATLAB with an LSTM classifier as 5-fold. The data is divided into 5 parts with 5 folds, and different data are divided into test and train in each cycle. In this way, all data is reviewed and accuracy is verified at each fold. In the end, the classification result is obtained by averaging the accuracy in 5 cycles. Among the 3 algorithms, with both transfer learning and 5-fold classification, the lowest result was obtained from the Alexnet pre-training algorithm with 94.3% accuracy, Vgg19 with 96% accuracy, and the highest result was obtained from resnet18 with 98% accuracy. The content of the study focuses on examining these truths in detail.

show abstract

“…Because cross-validation is necessary to obtain reliable and more accurate results. In each cycle, test and training data are randomly divided and this is done 5 times [20].…”

Section: Resultsmentioning

confidence: 99%

Comparison of Video Classification Results with Machine Learning

Akarsu,

Karacali

2024

View full text Add to dashboard Cite

show abstract

“…The attention-based model transformer is proved to have better performance than Long Short Term Memory (LSTM). Du et al (2022) emphasized the Spatio-temporal transformation of video understanding [12].…”

Section: Introductionmentioning

confidence: 99%

The Application of Deep Convolution Neural Network in Volleyball Video Behavior Recognition

Liang

2022

IEEE Access

View full text Add to dashboard Cite

The purpose is to solve the problems of solid subjectivity in the traditional manual analysis of volleyball game videos and the traditional Human Behavior Recognition (HBR) algorithm, such as excessive calculation, high hardware conditions, and poor long-stream video modeling ability. Firstly, this paper expounds on the relevant theories. Secondly, a fusion Convolutional Neural Network (CNN)-based HBR model is implemented by combining two-stream CNN (TSCNN), Three-Dimensional (3D) CNN, and Long Short-Term Memory (LSTM) network. Notably, the LSTM has an excellent long-term Dynamic Information Extraction (DIE) ability. Finally, the public dataset is selected to verify the model's volleyball-game-videooriented HBR performance. The experimental results corroborate that: (1) The parameters of the proposed fusion-CNN-based HBR model are determined as follows: the number of video segments is three, the average method is used for feature fusion, and then the HBR accuracy is the highest when the fusion ratio of spatial feature map and temporal feature map is 4:6, and the learning rate is 0.0014. (2) The average HBR accuracy of the proposed fusion-CNN-based HBR model on three different datasets is 4%, 2.7%, and 3% higher than that of other popular networks, respectively. The improvement effect of the model is remarkable, and it is suitable for the study of Human Behavior Analysis (HBA) in volleyball match videos. Finally, the proposed HBR model can provide more accurate results for volleyball video-based HBR, which is significant in promoting the rapid development of volleyball sports.

show abstract

Spatio-Temporal Transformer for Online Video Understanding

Cited by 2 publications

References 9 publications

Comparison of Video Classification Results with Machine Learning

Comparison of Video Classification Results with Machine Learning

The Application of Deep Convolution Neural Network in Volleyball Video Behavior Recognition

Contact Info

Product

Resources

About