DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents

Abstract

We introduce a Deep Stochastic IOC 1 RNN Encoderdecoder framework, DESIRE, for the task of future predictions of multiple interacting agents in dynamic scenes. DESIRE effectively predicts future locations of objects in multiple scenes by 1) accounting for the multi-modal nature of the future prediction (i.e., given the same context, future may vary), 2) foreseeing the potential future outcomes and make a strategic prediction based on that, and 3) reasoning not only from the past motion history, but also from the scene context as well as the interactions among the agents. DESIRE achieves these in a single end-to-end trainable neural network model, while being computationally efficient. The model first obtains a diverse set of hypothetical future prediction samples employing a conditional variational autoencoder, which are ranked and refined by the following RNN scoring-regression module. Samples are scored by accounting for accumulated future rewards, which enables better long-term strategic decisions similar to IOC frameworks. An RNN scene context fusion module jointly captures past motion histories, the semantic scene context and interactions among multiple agents. A feedback mechanism iterates over the ranking and refinement to further boost the prediction accuracy. We evaluate our model on two publicly available datasets: KITTI and Stanford Drone Dataset. Our experiments show that the proposed model significantly improves the prediction accuracy compared to other baseline methods.

Context-Based Path Prediction for Targets with Switching Dynamics

“…Neural nets can also learn how multiple agents move in each others presence Yi et al 2016), even from a vehicle perspective (Karasev et al 2016;Lee et al 2017). In the IV domain, interaction of road users with the ego-vehicle is especially important.…”

Planning and Decision-Making for Autonomous Vehicles

“…An alternative is to employ a nonparametric prediction architecture (107). A sample generation module consisting of a conditional variational autoencoder was able to learn a sampling model that, given observations of past trajectories, produces a diverse set of prediction hypotheses to capture the multimodality of the space of plausible futures.…”

Ambiguity in Sequential Data: Predicting Uncertain Futures With Recurrent Models

“…Encoder-decoder architectures excel in many sequential problems like trajectory prediction [1], [2]. Encoder-decoder architectures consist of two separate recurrent networks: the encoder processes the input sequence x i = (x i,1 , .…”

“…Additionally, we introduce a new metric tailored for ambiguous problems, as we found previous ones to be unsuited. Indeed, many previous works use some form of oracle metric, which only considers the best hypothesis [1], [2]. Although theoretically sound, this could easily be fooled by guessing a multitude of diverse solutions in hopes of approximating the correct one.…”