Adaptive most joint selection and covariance descriptions for a robust skeleton-based human action recognition

Nguyen, Van Duy; Nguyen, Tien-Nam; Le, Thi‐Lan; Pham, Dinh‐Tan; Vu, Hai

doi:10.1007/s11042-021-10866-4

Cited by 7 publications

(9 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In Ref. [17], the authors have shown a big gap between the performances of the cross‐dataset and benchmark dataset evaluation. This means that the performance of the methods can significantly decrease if the testing environment is different from the one that the methods have learnt.…”

Section: Methodsmentioning

confidence: 99%

“…To leverage the cross‐dataset evaluation, based on MSR‐Action3D [29], the authors in Ref. [17] have built a new dataset named MICA‐Action3D. MICA‐Action3D has the same list of actions and is divided into three subsets named AS1, AS2, and AS3 as Ref.…”

Section: Methodsmentioning

confidence: 99%

“…Although many datasets are built for skeleton-based HAR, the action sets differ from one dataset to another. To the best of our knowledge, two main studies prepare data for cross-dataset evaluation: [17,23].…”

Section: Dataset For Cross-dataset Evaluationmentioning

confidence: 99%

“…In Ref. [17], the authors propose combining both joint velocity covariance descriptors and joint position covariance descriptors extracted from MIJ for action recognition.…”

Section: Related Workmentioning

confidence: 99%

“…Although GCN models have obtained state-of-the-art results on different datasets, GCN models are computationally complex and subject to robustness limitations when working with small datasets [6,17]. Therefore, in Ref.…”

Section: Skeleton-based Human Activity Recognitionmentioning

confidence: 99%

See 4 more Smart Citations

A robust and efficient method for skeleton‐based human action recognition and its application for cross‐dataset evaluation

Nguyen

Pham

et al. 2022

IET Computer Vision

Self Cite

View full text Add to dashboard Cite

Skeleton-based human action recognition has emerged recently thanks to its compactness and robustness to appearance variations. Although impressive results have been obtained in recent years, the performance of skeleton-based action recognition methods has to be improved to be deployed in real-time applications. Recently, a lightweight network structure named Double-feature Double-motion Network (DD-Net) has been proposed for the skeleton-based human action recognition. With high speed, the DD-Net achieves state-of-the-art performance on hand and body actions. The DD-Net could not distinguish actions if they have a weak connection with the global trajectories. However, the DD-Net is suitable for human action recognition where actions strongly correlate to the global trajectories. In this paper, the authors propose TD-Net, an improved version of the DD-Net in which a new branch is added. The new branch takes the normalised coordinates of joints (NCJ) to enrich the spatial information. On five datasets for skeletonbased human activity recognition that are MSR-Action3D, CMDFall, JHMDB, FPHAB, and NTU RGB + D, the TD-Net consistently obtains superior performance compared with the baseline model DD-Net. The proposed method outperforms different state-ofthe-art methods, including both hand-designed and deep learning-based methods on four datasets (MSR-Action3D, CMDFall, JHMDB, and FPHAB). Furthermore, the generalisation of the proposed method is confirmed through cross-dataset evaluation. To illustrate the potential use of the model for real-time human action recognition, the authors have deployed an application on an edge device. The experimental result shows that the application can process up to 40 fps for pose estimation using MediaPipe. It takes only 0.04 ms to recognise an action from skeleton sequences.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Dataset For Cross-dataset Evaluationmentioning

confidence: 99%

“…In Ref. [17], the authors propose combining both joint velocity covariance descriptors and joint position covariance descriptors extracted from MIJ for action recognition.…”

Section: Related Workmentioning

confidence: 99%

Section: Skeleton-based Human Activity Recognitionmentioning

confidence: 99%

See 3 more Smart Citations