A Comprehensive Review of Group Activity Recognition in Videos

Abstract

Human group activity recognition (GAR) has attracted significant attention from computer vision researchers due to its wide practical applications in security surveillance, social role understanding and sports video analysis. In this paper, we give a comprehensive overview of the advances in group activity recognition in videos during the past 20 years. First, we provide a summary and comparison of 11 GAR video datasets in this field. Second, we survey the group activity recognition methods, including those based on handcrafted features and those based on deep learning networks. For better understanding of the pros and cons of these methods, we compare various models from the past to the present. Finally, we outline several challenging issues and possible directions for future research. From this comprehensive literature review, readers can obtain an overview of progress in group activity recognition for future studies.

COMPOSER: Compositional Reasoning of Group Activity in Videos with Keypoint-Only Modality

“…Early work on GAR relies on handcrafted features [13,15,17,18,31,53,66]; yet notable progress has been made in recent years by deep-learning (DL) based approaches [22,41]. We review DL-based methods and refer readers to the comprehensive review of GAR presented in [97]. Early DLbased methods use Convolutional Neural Networks (CNNs) to extract features and then apply recurrent neural networks for temporal modeling [46,58,80,95].…”

“…Early DLbased methods use Convolutional Neural Networks (CNNs) to extract features and then apply recurrent neural networks for temporal modeling [46,58,80,95]. Since learning interperson interactions is essential for GAR [97], much of the research explores how to capture the actor relations [4,36,40,72,96]. Several works tackle this problem from a graphbased perspective [40,63,100,101] such as applying Graph Convolutional Networks (GCNs) [49,96].…”

“…The Group Activity Recognition (GAR) task detects the activity performed by a group of actors interacting with one other in a short video clip [16,97]. GAR has wide-spread applications in sports analytics, robot-human interaction, Figure 1.…”

“…First, GAR requires a compositional understanding of the scene [1]. Because of the crowded scene, it is challenging to learn meaningful representations for GAR over the entire scene [97].…”

“…Since group activity often consists of one or more subgroups of actors and scene objects, the final action label depends on a compositional understanding of these entities [97,103]. Second, GAR benefits from relational reasoning over scene elements to understand the relative importance of entities and their interactions [36,101].…”