View-Adaptive Graph Neural Network for Action Recognition

Abstract

Skeleton-based recognition of human actions has received attention recently because of popular 3D acquisition sensors. Existing studies use skeleton data from video clips collected from several views. When humans perform certain actions, the body view shifts from the camera perspective, resulting in unstable and noisy skeletal data, and the possibility of self-occlusions makes it very challenging for recognition. To counteract the influence of variations, we developed a view-adaptive mechanism that identifies the viewpoints across the action video sequence and transforms the skeleton view through a data-driven learning process. Most existing methods use a fixed human-defined prior criterion to re-position the skeletons. In contrast, we utilize an unsupervised reposition approach and jointly design a view-adaptive (VA) neural network based on the graph neural network (GNN). Our VA-GNN model can transform the skeletons of distinct views into more consistent virtual perspective than comparative pre-processing approaches. The VA module learns the best-observed view because it determines the most suitable view and transforms the skeletons from action sequence for end-to-end recognition along with suited graph topology with adaptive GNN. Experiments on three datasets achieve a better accuracy and a lower number of parameters demonstrating the effectiveness of our approach.