Multimodal and multiview distillation for real-time player detection on a football field

Abstract

Monitoring the occupancy of public sports facilities is essential to assess their use and to motivate their construction in new places. In the case of a football field, the area to cover is large, thus several regular cameras should be used, which makes the setup expensive and complex. As an alternative, we developed a system that detects players from a unique cheap and wide-angle fisheye camera assisted by a single narrow-angle thermal camera. In this work, we train a network in a knowledge distillation approach in which the student and the teacher have different modalities and a different view of the same scene. In particular, we design a custom data augmentation combined with a motion detection algorithm to handle the training in the region of the fisheye camera not covered by the thermal one. We show that our solution is effective in detecting players on the whole field filmed by the fisheye camera. We evaluate it quantitatively and qualitatively in the case of an online distillation, where the student detects players in real time while being continuously adapted to the latest video conditions.

Feature Combination Meets Attention: Baidu Soccer Embeddings and Transformer based Temporal Detection

“…In sports analytics, many computer vision technologies are developed to understand sports broadcasts [15]. Specifically in soccer, researchers propose algorithms to detect players on field in real time [2], analyze pass feasibility using player's body orientation [1], incorporate both audio and video streams to detect events [17], recognize group activities on the field using broadcast stream and trajectory data [14], aggregate deep frame features to spot major game events [8], and leverage the temporal context information around the actions to handle the intrinsic temporal patterns representing these actions [3,9].…”

Detecting and Matching Related Objects with One Proposal Multiple Predictions

“…Object detection has been widely used in automated tracking systems for sports [18,12,4,19]. The task poses additional challenges to generic object detection, including camera distortion and fast movement (for broadcast videos), as well as frequent crowded scenes.…”

“…Acuna [1] introduces an end-to-end CNN-based object detector in the basketball. In order to improve the detection result for fisheye cameras in soccer, Cioppa et al [4] applies a single stage object detector on cropped and rotated image patches.…”

“…Detecting players is the foundation of automated player tracking in sport games [18,12,4], in a tracking-bydetection framework [2,21,20]. The detection task normally focuses on finding bounding boxes around players and, for some applications, a box around the ball.…”

DeepDarts: Modeling Keypoints as Objects for Automatic Scorekeeping in Darts using a Single Camera

“…Deep learning-based computer vision has recently gained traction in the sports industry due to its ability to autonomously extract data from sports video feeds that would otherwise be too tedious or expensive to collect manually. Example sports applications include field localization [20], player detection and tracking [39,11,42], equipment and object tracking [45,44,36], pose estimation [5,29,2], event detection [27,15,6,43,37], and scorekeeping [45]. This data is often more informative than conventional humanrecorded statistics and as such, the technology is quickly ushering in a new era of sports analytics.…”