PRECOG: PREdiction Conditioned on Goals in Visual Multi-Agent Settings

Abstract

For autonomous vehicles (AVs) to behave appropriately on roads populated by human-driven vehicles, they must be able to reason about the uncertain intentions and decisions of other drivers from rich perceptual information. Towards these capabilities, we present a probabilistic forecasting model of future interactions of multiple agents. We perform both standard forecasting and conditional forecasting with respect to the AV's goals. Conditional forecasting reasons about how all agents will likely respond to specific decisions of a controlled agent. We train our model on real and simulated data to forecast vehicle trajectories given past positions and LIDAR. Our evaluation shows that our model is substantially more accurate in multi-agent driving scenarios compared to existing state-of-the-art. Beyond its general ability to perform conditional forecasting queries, we show that our model's predictions of all agents improve when conditioned on knowledge of the AV's intentions, further illustrating its capability to model agent interactions.

Argoverse: 3D Tracking and Forecasting With Rich Maps

“…R2P2 [50] address the diversity-precision trade-off of generative forecasting models and formulate a symmetric cross-entropy training objective to address it. It is then followed by PRECOG [51] wherein they present the first generative multi-agent forecasting method to condition on agent intent. They achieve state-of-the-art results for forecasting methods in real (nuScenes [6]) and simulated (CARLA [12]) datasets.…”

“…In the case of autonomous vehicles, it is important to predict many plausible outcomes and not simply the most likely outcome. While some prior works have evaluated forecasting in a deterministic, unimodal way, we believe a better approach is to follow the evaluation methods similar to DESIRE [32], Social GAN [17], R2P2 [50] and [51] wherein they encourage algorithms to output multiple predictions. Among the variety of metrics evaluated in [50] was the minMSD over K number of samples metric, where K = 12.…”

“…That said, minADE error might not be a sufficient metric. As noted in [50] and [51], metrics like minMSD or minFDE can only evaluate how good is the best trajectory, but not how good are all the trajectories. A model having 5 good trajectories will have the same error as the model having 1 good and 4 bad trajectories.…”

Multiple Trajectory Prediction of Moving Agents With Memory Augmented Networks

“…Traffic dynamics likely reduce to simpler scenarios where movement is limited and constrained by the environment. Efforts have been made to understand and predict vehicle trajectories in urban scenarios [40], [51], [52], [53], [54], [55], [56], [57], [58], [59], also taking into account social interactions. Although, from the empirical evidence presented in [11], [51], the explicit modeling of social interactions for vehicles was shown not to provide valuable improvements in trajectory prediction.…”

“…A conceptually similar research direction to ours is the one of intention-based methods [54], [55], [56]. In these works, some representative anchor information (such as trajectories, actions or locations) are predefined and then used to guide predictions after estimating a probability distribution over each candidate.…”