Interaction Classification with Key Actor Detection in Multi-Person Sports Videos

Askari, Farzaneh; Ramaprasad, Rohit; Clark, James J.; Levine, Martin D.

doi:10.1109/cvprw56347.2022.00402

Cited by 6 publications

(3 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The skeleton joints vector sequences are previously unexplored pose information and assign greater significance attributed to individuals who are more relevant for distinguishing the group activity behavior. Evaluate the Hockey dataset, Askari et al [ 21 ] reports 93.93% accuracy using RNN equipped with a time-varying attention mechanism, our baseline architectures, 𝐴𝑅𝑁 𝑖𝑛𝑡𝑒𝑟 and 𝐴𝑅𝑁 𝑖𝑛𝑡𝑟𝑎 , achieved accuracies of 92.14% and 95.36% respectively, demonstrating the successful mapping of various relationships in the skeleton-based group activity recognition problem. Then, we attempted to fuse the two models by simply averaging their scores, but proved ineffective, yielding lower performance than only 𝐴𝑅𝑁 𝑖𝑛𝑡𝑒𝑟 .…”

Section: Ablation Studymentioning

confidence: 79%

“…The hockey penalty dataset [ 21 ] includes five Slashing, Holding, Tripping, Hooking, and No Penalty classes, with 76, 80, and 98 clips, respectively. The multi-person videos in this study depict intricate interactions between players within a non-laboratory recording setup.…”

Section: A Datasetsmentioning

confidence: 99%

“…We validate our approach through experiments on a traditional human interaction recognition dataset: The hockey [ 21 ] and UT-interaction [ 22 ] datasets. Our proposed solution demonstrates competitive performance to the stateof-the-art on these mutual action datasets.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Attention Relational Network for Skeleton-Based Group Activity Recognition

Wang,

Mohamed

2023

IEEE Access

View full text Add to dashboard Cite

Group Activity Recognition refers to multi-individual activity analysis. It is a significant and challenging task in computer vision. The solution of group activity prediction can be classified with traditional hand-crafted features, RGB video features, and skeleton data-based deep learning architectures, such as Graph Convolutional Networks (GCNs), Recurrent Neural Networks (RNNs), and Long Short-Term Memory (LSTMs). However, not only are those solutions usually designed to be complex, but they also rarely explore pose information and rarely use relational networks to reason about group activity behavior. They disregard the magnitude and orientation details of the skeletal edges, crucial for action recognition, potentially leading to suboptimal outcomes in these methodologies. In this work, we leverage minimal prior knowledge about the skeleton information to reason about the interactions from group activity. The objective is to obtain discriminative representations and filter out some ambiguous actions to enhance the performance of group activity recognition. Our contribution is a proposed Attention Relation Network (ARN) that fuses the attention mechanisms and joint vector sequences into the relation network. The skeleton joints vector sequences are previously unexplored pose information and assign greater significance attributed to individuals who are more relevant for distinguishing the group activity behavior. In the first stage, our model focuses on the specified edge-level information (encompassing both edge and edge motion data) within the skeleton dataset, considering directionality, to analyze the spatiotemporal aspects of the action. In the second stage, recognizing the inherent motion directionality, we establish diverse directions for skeleton edges and extract distinct motion features (including translation and rotation information) aligned with these various orientations, thereby augmenting the utilization of motion attributes related to the action. We also introduce a representation of human motion achieved by combining relational networks and examining their integrated characteristics. Extensive experiments were tested in the Hockey and UT-interaction datasets to evaluate our method, obtaining competitive performance to the state-of-the-art. Results demonstrate the modeling potential of a skeletonbased method for group activity recognition.

show abstract

Section: Ablation Studymentioning

confidence: 79%

Section: A Datasetsmentioning

confidence: 99%

See 1 more Smart Citation

Attention Relational Network for Skeleton-Based Group Activity Recognition

Wang,

Mohamed

2023

IEEE Access

View full text Add to dashboard Cite

show abstract

Motion-aware and data-independent model based multi-view 3D pose refinement for volleyball spike analysis

LIU

Cheng

Ikenaga

2023

Multimed Tools Appl

View full text Add to dashboard Cite

In the volleyball game, estimating the 3D pose of the spiker is very valuable for training and analysis, because the spiker’s technique level determines the scoring or not of a round. The development of computer vision provides the possibility for the acquisition of the 3D pose. Most conventional pose estimation works are data-dependent methods, which mainly focus on reaching a high level on the dataset with the controllable scene, but fail to get good results in the wild real volleyball competition scene because of the lack of large labelled data, abnormal pose, occlusion and overlap. To refine the inaccurate estimated pose, this paper proposes a motion-aware and data-independent method based on a calibrated multi-camera system for a real volleyball competition scene. The proposed methods consist of three key components: 1) By utilizing the relationship of multi-views, an irrelevant projection based potential joint restore approach is proposed, which refines the wrong pose of one view with the other three views projected information to reduce the influence of occlusion and overlap. 2) Instead of training with a large amount labelled data, the proposed motion-aware method utilizes the similarity of specific motion in sports to achieve construct a spike model. Based on the spike model, joint and trajectory matching is proposed for coarse refinement. 3) To finely refine, a point distribution based posterior decision network is proposed. While expanding the receptive field, the pose estimation task is decomposed into a classification decision problem, which greatly avoids the dependence on a large amount of labelled data. The experimental dataset videos with four synchronous camera views are from a real game, the Game of 2014 Japan Inter High School of Men Volleyball. The experiment result achieves 76.25%, 81.89%, and 86.13% success rate at the 30mm, 50mm, and 70mm error range, respectively. Since the proposed refinement framework is based on a real volleyball competition, it is expected to be applied in the volleyball analysis.

show abstract

Self-Supervised Video Interaction Classification using Image Representation of Skeleton Data

Askari

Jiang

et al. 2023

2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

View full text Add to dashboard Cite

Interaction Classification with Key Actor Detection in Multi-Person Sports Videos

Cited by 6 publications

References 39 publications

Attention Relational Network for Skeleton-Based Group Activity Recognition

Attention Relational Network for Skeleton-Based Group Activity Recognition

Motion-aware and data-independent model based multi-view 3D pose refinement for volleyball spike analysis

Self-Supervised Video Interaction Classification using Image Representation of Skeleton Data

Contact Info

Product

Resources

About