HFNet: A Novel Model for Human Focused Sports Action Recognition

Abstract

Action recognition has attracted much attention recently and progressed remarkably. However, as a special kind of actions, sports action recognition is more difficult and deserves more attention. Our goal in this paper is to distinguish fine-grained human-focused sport actions. Sport actions can always be decomposed into subactions by body parts and it's necessary to establish the relationships among body parts and combine them together to perform classification. Besides, sport actions are usually fine-grained and subclasses a re similar which are hard to distinguish. Another tough problem in practice is to locate the actor in complicated circumstances. However, current methods in action recognition always pay attention to the whole image, thus failing to capture details and constructing relationships in images. In this paper, we propose a novel model to construct visual relationships in images through graph convolutions. We make use of patches cropped around body joints as input for graph nodes. Thus our model is able to pay attention to the changes and details of body parts. Then, we carefully design model to learn connections among graph nodes adaptively and empirically. We also provide another method to construct visual relationships for graph nodes. By specially focusing on relationships and details, our model achieves start-of-the-art performance on complex human-focused sports datasets FSD-10 and Diving48. CCS CONCEPTS • Computing methodologies → Activity recognition and understanding; Scene understanding; Motion capture.

Fully Attentional Network for Skeleton-Based Action Recognition

“…A self-attention module flexibly computes the response of a position with others' in an embedding space and then aggregates their weighted features. Self-attention mechanism has proven its power in many fields including machine reading [45], learning sentence representations [46] and especially video action recognition [3], [47] which is closely related to us. It has also been combined with graph networks in inductive or transductive learning [48] and further introduced to model action sequences [28], [31], [32], [35], [40], [43], [44], [47].…”

“…Self-attention mechanism has proven its power in many fields including machine reading [45], learning sentence representations [46] and especially video action recognition [3], [47] which is closely related to us. It has also been combined with graph networks in inductive or transductive learning [48] and further introduced to model action sequences [28], [31], [32], [35], [40], [43], [44], [47]. Although self-attention mechanism has been introduced into skeleton-based action recognition, we explain next that these methods haven't fully released its potential.…”