Semantics-enhanced algorithm for skeleton-based action recognition

Abstract

Skeleton-based action recognition has attracted the attention of many researchers. In the process of extracting skeleton features, most methods use the first-order features (the joint position of the skeleton) or the second-order features (the length and direction of the skeleton) to represent the skeleton, while ignoring the importance of skeleton semantic features to action. In this paper, a new skeleton appearance semantic feature is proposed to describe the appearance feature of human skeleton. An attention mechanism of skeleton appearance semantic features is further proposed, which integrates channel features to highlight the overall difference. In addition, a method is used to integrate the semantic features of skeleton appearance with the position and velocity features of skeleton joints, as well as the constructed joints type and frame index, so as to improve the representation method of skeleton semantic features. The experiments were carried out in two public skeleton action recognition data sets (NTU-RGB+D 60 and NTU-RGB+D 120), and the recognition accuracy was higher than that of the baseline model SGN.