Temporally Precise Action Spotting in Soccer Videos Using Dense Detection Anchors

Abstract

We present a model for temporally precise action spotting in videos, which uses a dense set of detection anchors, predicting a detection confidence and corresponding fine-grained temporal displacement for each anchor. We experiment with two trunk architectures, both of which are able to incorporate large temporal contexts while preserving the smaller-scale features required for precise localization: a one-dimensional version of a u-net, and a Transformer encoder (TE). We also suggest best practices for training models of this kind, by applying Sharpness-Aware Minimization (SAM) and mixup data augmentation. We achieve a new state-of-the-art on SoccerNet-v2, the largest soccer video dataset of its kind, with marked improvements in temporal localization. Additionally, our ablations show: the importance of predicting the temporal displacements; the trade-offs between the u-net and TE trunks; and the benefits of training with SAM and mixup.

On the completion of automatic football game commentary system with deep learning

“…Results &Comparison with baselines and state-of-the-art: Table 1 shows the performance of different methods of the SoccerNet challenge. Among of different proposed methods, our model achieve a comparable result among other approaches [6,7,9] in tight unshown average mAP as 53.80%, which is the leading position in unshown average mAP metric of the SoccerNet challenge 2022, excepting for [8]. Our objective is eventually to form a real-time system for automatic football commentary, hence real-time key action spotting is also our final objective.…”

“…Our objective is eventually to form a real-time system for automatic football commentary, hence real-time key action spotting is also our final objective. Let us compare our approach and [8] which used a much larger chunk size, 112s, for prediction. Although this approach is beneficial to accuracy and hence increased the average mAP, this approach sacrifices the adaptability to detect actions during real-time broadcast.…”

“…Although this approach is beneficial to accuracy and hence increased the average mAP, this approach sacrifices the adaptability to detect actions during real-time broadcast. In contrast, our approach only requires 3s chunk size for prediction, which is the smallest chunk size among [5][6][7][8][9]. Due to smallest chunk size required, this approach has a better chance to become a real-time implementation of automatic commentary generation.…”