An Application of Transfer to American Football: From Observation of Raw Video to Control in a Simulated Environment

Abstract

Automatic transfer of learned knowledge from one task or domain to another offers great potential to simplify and expedite the construction and deployment of intelligent systems. In practice however, there are many barriers to achieving this goal. In this article, we present a prototype system for the real-world context of transferring knowledge of American football from video observation to control in a game simulator. We trace an example play from the raw video through execution and adaptation in the simulator, highlighting the system's component algorithms along with issues of complexity, generality, and scale. We then conclude with a discussion of the implications of this work for other applications, along with several possible improvements.

Large-Scale Analysis of Soccer Matches Using Spatiotemporal Tracking Data

“…Bricola [25] recognized activities in basketball from player trajectories by segmented the trajectories into tracklets which were matched to codewords using Dynamic Time Warping. Stracuzzi et al [26] recognized group activities in American Football using a labeled dataset of actions and the trajectories were labeled by matching them to the closest in the labeled dataset. Dynamic time warping was used to compare the signals and the features of each aligned point.…”

Representing and Discovering Adversarial Team Behaviors Using Player Roles

“…Li and Chellapa [20] used a spatio-temporal driving force model to segment the two groups/teams using their trajectories. Researchers at Oregon State University have looked at automatically detecting offensive plays from raw video and transfer this knowledge to a simulator [29]. For soccer, Kim et al [16] used the global motion of all players in a soccer match to predict where the play will evolve in the short-term.…”

Hybrid Multi-Agent Strategy Discovering Algorithm for human behavior

“…We, on the other hand, do not use predefined behavior libraries, and do not provide counter-strategies. Stracuzzi et al (2012) demonstrated a knowledge transfer method by observing videos of American-football in order to control agent players in a game simulator. It included object tracking, dynamic time warping for action recognition, and an Icarus cognitive architecture to infer strategies in the form of rules.…”