The Trajectron: Probabilistic Multi-Agent Trajectory Modeling With Dynamic Spatiotemporal Graphs

Abstract

Developing safe human-robot interaction systems is a necessary step towards the widespread integration of autonomous agents in society. A key component of such systems is the ability to reason about the many potential futures (e.g. trajectories) of other agents in the scene. Towards this end, we present the Trajectron, a graph-structured model that predicts many potential future trajectories of multiple agents simultaneously in both highly dynamic and multimodal scenarios (i.e. where the number of agents in the scene is time-varying and there are many possible highlydistinct futures for each agent). It combines tools from recurrent sequence modeling and variational deep generative modeling to produce a distribution of future trajectories for each agent in a scene. We demonstrate the performance of our model on several datasets, obtaining state-of-the-art results on standard trajectory prediction metrics as well as introducing a new metric for comparing models that output distributions.

Human motion trajectory prediction: a survey

“…Other approaches used RNN as models of spatiotemporal graphs for problems that require both spatial and temporal reasoning (Dai et al, 2019; Eiffert and Sukkarieh, 2019; Huang et al, 2019; Ivanovic and Pavone, 2019; Jain et al, 2016; Vemula et al, 2018). Jain et al (2016) proposed an approach for training sequence prediction models on arbitrary high-level spatiotemporal graphs, whose nodes and edges are represented by RNNs.…”

Multiple Trajectory Prediction of Moving Agents With Memory Augmented Networks

“…Trajectory Prediction Significant effort has been made in the past years regarding trajectory prediction. Several researchers have focused on trajectories of pedestrians [6], [7], [46], [47], [48], either regarded as individuals or crowds, also exploiting social behaviors and interactivity between individuals [6], [7], [46], [47], [49], [50].…”

Multi-Modal Pedestrian Trajectory Prediction for Edge Agents Based on Spatial-Temporal Graph

“…However, due to the uncertainty and randomness of human activities, the multimodal prediction trajectory is more consistent with the real situation. In the latest research [10], [23], [41], [42], researchers have considered the multi-modality of pedestrian trajectory prediction. These methods are based on generative adversarial networks, which consist of two competing networks, namely a generator and a discriminator.…”

“…The input of the STG-GAN prediction model is the observable scene context information, including the historical trajectory of all pedestrians and the location of fixed obstacles. Similar to the definition in [10], [21], [23], where the observable historical trajectory of pedestrian i is defined as:…”

“…Pedestrian trajectory prediction in dynamic scenes is a challenging task, which requires simultaneous consideration of the spatial-temporal structure of human-environment interaction, visual attention [9] and multi-modal behavior [10] of human walking. In a dynamic scene, when making a navigation decision, the person will consider the location of the fixed obstacle and the dynamic participants in the scene to adjust their own path to avoid collision.…”