CAR-Net: Clairvoyant Attentive Recurrent Network

Abstract

We present an interpretable framework for path prediction that leverages dependencies between agents' behaviors and their spatial navigation environment. We exploit two sources of information: the past motion trajectory of the agent of interest and a wide top-view image of the navigation scene. We propose a Clairvoyant Attentive Recurrent Network (CAR-Net) that learns where to look in a large image of the scene when solving the path prediction task. Our method can attend to any area, or combination of areas, within the raw image (e.g., road intersections) when predicting the trajectory of the agent. This allows us to visualize fine-grained semantic elements of navigation scenes that influence the prediction of trajectories. To study the impact of space on agents' trajectories, we build a new dataset made of top-view images of hundreds of scenes (Formula One racing tracks) where agents' behaviors are heavily influenced by known areas in the images (e.g., upcoming turns). CAR-Net successfully attends to these salient regions. Additionally, CAR-Net reaches state-of-the-art accuracy on the standard trajectory forecasting benchmark, Stanford Drone Dataset (SDD). Finally, we show CAR-Net's ability to generalize to unseen scenes.

Multi-Agent Tensor Fusion for Contextual Trajectory Prediction

“…Early methods based on inverse optimal control also use hand-crafted cost features, and learn linear weighting functions to rationalize trajectories which are assumed to be generated by optimal control [18]. Recent data-driven approaches based on deep networks [1,4,9,10,13,19,20,24,28,29,31] outperform traditional approaches. Most of this work focuses either on modeling constraints from the scene context [29] or on modeling social interactions among multiple agents [1,9,10,13,31]; a smaller fraction of work considers both aspects [4,20,28].…”

“…Many data-driven approaches learn to predict deterministic future trajectories of agents by minimizing reconstruction loss [1,29]. However, human behavior is inherently stochastic.…”

Looking to Relations for Future Trajectory Forecast

“…Such technologies require advanced decision making and motion planning systems that rely on estimates of the future position of road users in order to realize safe and effective mitigation and navigation strategies. Related research [46,1,36,23,37,12,13,43,45,32,33,47] has attempted to predict future trajectories by focusing on social conventions, environmental factors, or pose and motion constraints. They have shown to be more effective when the prediction model learns to extract these features by considering human-human (i.e., between road agents) or human-space (i.e., between a road agent and environment) interactions.…”

Social and Scene-Aware Trajectory Prediction in Crowded Spaces

“…Similarly, in [6] circular distributions model dynamics and semantics for long-term trajectory predictions. [19] uses past observations along with bird's eye view images based on a two-levels attention mechanism. The work mainly focuses on scene cues partially addressing agents' interactions.…”