From the Laboratory to the Field: IMU-Based Shot and Pass Detection in Football Training and Game Scenarios Using Deep Learning

Abstract

The applicability of sensor-based human activity recognition in sports has been repeatedly shown for laboratory settings. However, the transferability to real-world scenarios cannot be granted due to limitations on data and evaluation methods. On the example of football shot and pass detection against a null class we explore the influence of those factors for real-world event classification in field sports. For this purpose we compare the performance of an established Support Vector Machine (SVM) for laboratory settings from literature to the performance in three evaluation scenarios gradually evolving from laboratory settings to real-world scenarios. In addition, three different types of neural networks, namely a convolutional neural net (CNN), a long short term memory net (LSTM) and a convolutional LSTM (convLSTM) are compared. Results indicate that the SVM is not able to reliably solve the investigated three-class problem. In contrast, all deep learning models reach high classification scores showing the general feasibility of event detection in real-world sports scenarios using deep learning. The maximum performance with a weighted f1-score of 0.93 was reported by the CNN. The study provides valuable insights for sports assessment under practically relevant conditions. In particular, it shows that (1) the discriminative power of established features needs to be reevaluated when real-world conditions are assessed, (2) the selection of an appropriate dataset and evaluation method are both required to evaluate real-world applicability and (3) deep learning-based methods yield promising results for real-world HAR in sports despite high variations in the execution of activities.

Wearable Sensors for Activity Recognition in Ultimate Frisbee Using Convolutional Neural Networks and Transfer Learning

“…Future work has to investigate whether the proposed pipeline works under competitive conditions during a match. Stoeve et al [ 41 ] investigated the transferability of a football activity recognition pipeline from controlled conditions to real-world scenarios. They observed a decrease in performance for a feature-based approach, whereas performance was comparable for the proposed CNN, thus indicating good generalization to complex scenarios [ 41 ].…”

Automatic Detection Algorithm of Football Events in Videos

“…Usually, two probability matrices are used to assist in describing the Hidden Markov Model (HMM). One is used to generate the state sequence (Markov chain), the other is used to constrain the observation sequence, and an initial distribution of the generated Markov chain is needed, which represents the distribution law of each hidden state when the Markov chain tends to be stable [ 22 ]. Let M be the size of the observation space, the observation element set is V = V 1 V 2 V 3 ⋯ V M , N is the size of the state space, and the state set element is S = S 1 S 2 S 3 ⋯ S N .…”

A Survey of Human Gait-Based Artificial Intelligence Applications

“…CNN and LSTM perform better than SVM (previously suggested in the literature) in the football shot and pass detection task in three scenarios closer to the real-world setting. The integrity of the collected data, selected features, and evaluation method must be reconsidered once AI systems are deployed in the real world ( Stoeve et al, 2021 ). Estimating kinematic data (that would usually be collected in a lab, using force plates) from kinetic data that is easily measured in the field using IMU sensors is the focus of the study by ( Johnson et al, 2021 ).…”