Key frame and skeleton extraction for deep learning-based human action recognition

Phan, Hai-Hong; Nguyen, Trung Tin; Phuc, Ngo Huu; Nhân, Nguyễn Hữu; Hieu, Do Minh; Tran, Cao Truong; Ngoc, Bao

doi:10.1109/rivf51545.2021.9642132

Cited by 2 publications

(2 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several researchers have proposed various keyframe extraction methods using different strategies. Phan et al [32] introduced an efficient framework named KFSENet for action recognition in videos, incorporating keyframe extraction based on skeleton deep learning architectures. Kim et al [33] proposed a bidirectional consecutively connected two pathway network (BCCN) for efficient gesture recognition using a Skeleton-Based Keyframe Selection Module.…”

Section: Keyframe Extractionmentioning

confidence: 99%

Skeleton Based Keyframe Detection Framework for Sports Action Analysis: Badminton Smash Case

Sarwar,

Lin,

Daraghmi

et al. 2023

IEEE Access

View full text Add to dashboard Cite

The analysis of badminton player actions from videos plays a crucial role in improving athletes' performance and generating statistical insights. The complexity and speed of badminton movements pose unique challenges compared to everyday activities. To analyze badminton player actions, we propose a skeleton-based keyframe detection framework for action analysis. Keyframe detection is widely used in video summarization and visual localization due to its computational efficiency and memory optimization compared to analyzing all frames of a video. This framework segments the complex macro-level activity into micro-level segments and analyzes each micro-level activity individually. Firstly, it extracts skeleton data from a motion sequence video using 3D:VIBE pose estimation. Then, the keyframe detection module explores the sequence of activity frames and identifies keyframes for each micro-level activity, including start, ready, strike, and end. Finally, the posture and movement detection modules analyze the posture and movement data to identify specific activities. This framework is implemented in the device called CoachBox. The proposed framework is evaluated using the mean absolute error on a dataset. The average mean absolute error for the keyframe detection module is less than 0.168 seconds, and the striking moment detection has an error of only 0.033 seconds. Additionally, a coordinate transform method is provided to convert body coordinates to real-world coordinates for visualization purposes.

show abstract

Section: Keyframe Extractionmentioning

confidence: 99%

Skeleton Based Keyframe Detection Framework for Sports Action Analysis: Badminton Smash Case

Sarwar,

Lin,

Daraghmi

et al. 2023

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Finding features from a smaller number of frames can reduce the computational complexity of the system because frames in videos have replicated activities also. The method [20] used a deep neural network and suggested selecting a keyframe. The LSTM is often applied [21] differential LSTM, which proposes feature enhancement.…”

Section: Introductionmentioning

confidence: 99%

Action Recognition for Multiview Skeleton 3D Data Using NTURGB + D Dataset

Bhogal,

Devendran

2023

CSSE

View full text Add to dashboard Cite

Human activity recognition is a recent area of research for researchers. Activity recognition has many applications in smart homes to observe and track toddlers or oldsters for their safety, monitor indoor and outdoor activities, develop Tele immersion systems, or detect abnormal activity recognition. Three dimensions (3D) skeleton data is robust and somehow view-invariant. Due to this, it is one of the popular choices for human action recognition. This paper proposed using a transversal tree from 3D skeleton data to represent videos in a sequence. Further proposed two neural networks: convolutional neural network recurrent neural network_1 (CNN_RNN_1), used to find the optimal features and convolutional neural network recurrent neural network network_2 (CNN_RNN_2), used to classify actions. The deep neural network-based model proposed CNN_RNN_1 and CNN_RNN_2 that uses a convolutional neural network (CNN), Long short-term memory (LSTM) and Bidirectional Long shortterm memory (BiLSTM) layered. The system efficiently achieves the desired accuracy over state-of-the-art models, i.e., 88.89%. The performance of the proposed model compared with the existing state-of-the-art models. The NTURGB + D dataset uses for analyzing experimental results. It is one of the large benchmark datasets for human activity recognition. Moreover, the comparison results show that the proposed model outperformed the state-ofthe-art models.

show abstract

Key frame and skeleton extraction for deep learning-based human action recognition

Cited by 2 publications

References 20 publications

Skeleton Based Keyframe Detection Framework for Sports Action Analysis: Badminton Smash Case

Skeleton Based Keyframe Detection Framework for Sports Action Analysis: Badminton Smash Case

Action Recognition for Multiview Skeleton 3D Data Using NTURGB + D Dataset

Contact Info

Product

Resources

About