Multiagent off-screen behavior prediction in football

Abstract

In multiagent worlds, several decision-making individuals interact while adhering to the dynamics constraints imposed by the environment. These interactions, combined with the potential stochasticity of the agents’ dynamic behaviors, make such systems complex and interesting to study from a decision-making perspective. Significant research has been conducted on learning models for forward-direction estimation of agent behaviors, for example, pedestrian predictions used for collision-avoidance in self-driving cars. In many settings, only sporadic observations of agents may be available in a given trajectory sequence. In football, subsets of players may come in and out of view of broadcast video footage, while unobserved players continue to interact off-screen. In this paper, we study the problem of multiagent time-series imputation in the context of human football play, where available past and future observations of subsets of agents are used to estimate missing observations for other agents. Our approach, called the Graph Imputer, uses past and future information in combination with graph networks and variational autoencoders to enable learning of a distribution of imputed trajectories. We demonstrate our approach on multiagent settings involving players that are partially-observable, using the Graph Imputer to predict the behaviors of off-screen players. To quantitatively evaluate the approach, we conduct experiments on football matches with ground truth trajectory data, using a camera module to simulate the off-screen player state estimation setting. We subsequently use our approach for downstream football analytics under partial observability using the well-established framework of pitch control, which traditionally relies on fully observed data. We illustrate that our method outperforms several state-of-the-art approaches, including those hand-crafted for football, across all considered metrics.

Ball Trajectory Inference from Multi-Agent Sports Contexts Using Set Transformer and Hierarchical Bi-LSTM

“…• Performance of the generative baseline (VRNN): Note that when the latent vector z 𝑡 of VRNN is sampled from the encoder 𝑞 𝜙 , the mean position error is less than 1 m. Nevertheless, the prediction using z 𝑡 sampled from the prior 𝑝 𝜃 is far worse than LSTM or Transformer baselines. The reason we think is that unlike the previous studies for trajectory prediction or imputation in team sports [19,37,52,54], the prior in our problem cannot leverage any fragmentary trajectory of the target. This hinders the model from reducing the KL divergence between 𝑝 𝜃 and 𝑞 𝜙 .…”

“…Also, Qi et al [41] proposed an imitative non-autoregressive modeling method to simultaneously handle the trajectory prediction task and the missing value imputation task. Omidshafiei et al [37] introduced Graph Imputer for predicting players' off-screen behavior in soccer, where the model is similar to GVRNN in that it combines graph networks and the Variational RNN [8], except for using a bidirectional structure since it can observe partial future trajectories.…”

“…In Section 4.2, we implement a generative baseline and compare the prediction performance with our regression model to explain our design choices. Since major studies [19,37,52,54] for player trajectory prediction or imputation have built their framework on top of Variational Recurrent Neural Network (VRNN) [8], we also construct a generative model based on the VRNN.…”

Applications of Representation Learning Methods in Professional Baseball

“…While a small number of works have sought to model the game as a sequence of events 15,16 , they usually had very narrow objectives, and the low-level, contextual understanding of the game is disregarded once the objective is achieved. Omidshafiei, et al 17 , for example, train RNN-based models to understand the movement of players and the ball around the soccer pitch with the goal of imputing player movement when not in frame of the broadcast camera. Knowledge of player location on the pitch is then used to quantify the amount of control each team has over the field of play.…”

TacticAI: an AI assistant for football tactics

“…Several recent methods attempt to improve tactical coaching and player decision-making through artificial intelligence (AI) tools, using a wide variety of data types from videos to tracking sensors and applying diverse algorithms ranging from simple logistic regression to elaborate neural network architectures. Such methods have been employed to help predict shot events from videos 3 , forecast off-screen movement from spatio-temporal data 4 , determine whether a match is in-play or interrupted 5 , or identify player actions 6 .…”