r2p2: A ReparameteRized Pushforward Policy for Diverse, Precise Generative Path Forecasting

Rhinehart, Nicholas; Kitani, Kris M.; Vernaza, Paul

doi:10.1007/978-3-030-01261-8_47

Cited by 205 publications

(226 citation statements)

References 32 publications

Supporting

Mentioning

225

Contrasting

Order By: Relevance

“…The "-" symbol indicates if an approach cannot compute likelihoods. The R2P2-MA generalizes the single-agent forecasting approach of [29]. Variants of our ESP method (highlighted gray) mostly outperform prior work in the multi-agent CARLA setting.…”

Section: E Carla Dataset Detailsmentioning

confidence: 96%

“…3. Finally, we performed additional featurization in the nuScenes setting by replacing χ with a signed-distance transform, similar to [29]. It provides a spatially-smoother input to the convolutional network, which we found augmented performance.…”

Section: Architecture and Training Detailsmentioning

confidence: 99%

“…Performance in CARLA in the single-agent setting. For single agent forecasting, our model is identical to[29] (denoted by a * ).…”

mentioning

confidence: 99%

See 2 more Smart Citations

PRECOG: PREdiction Conditioned on Goals in Visual Multi-Agent Settings

Rhinehart

McAllister

Kitani

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

Self Cite

322

236

View full text Add to dashboard Cite

For autonomous vehicles (AVs) to behave appropriately on roads populated by human-driven vehicles, they must be able to reason about the uncertain intentions and decisions of other drivers from rich perceptual information. Towards these capabilities, we present a probabilistic forecasting model of future interactions of multiple agents. We perform both standard forecasting and conditional forecasting with respect to the AV's goals. Conditional forecasting reasons about how all agents will likely respond to specific decisions of a controlled agent. We train our model on real and simulated data to forecast vehicle trajectories given past positions and LIDAR. Our evaluation shows that our model is substantially more accurate in multi-agent driving scenarios compared to existing state-of-the-art. Beyond its general ability to perform conditional forecasting queries, we show that our model's predictions of all agents improve when conditioned on knowledge of the AV's intentions, further illustrating its capability to model agent interactions.

show abstract

Section: E Carla Dataset Detailsmentioning

confidence: 96%

Section: Architecture and Training Detailsmentioning

confidence: 99%

See 1 more Smart Citation

PRECOG: PREdiction Conditioned on Goals in Visual Multi-Agent Settings

Rhinehart

McAllister

Kitani

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

Self Cite

322

236

View full text Add to dashboard Cite

show abstract

“…We seek to model an entity's future states. We believe a good output representation must have the following characteristics, which differs from some previous work [10,31]. It should be: (1) A probability distribution over the entity state space at each timestep.…”

Section: Output Representation: Modeling Uncertainty and Multiple Modesmentioning

confidence: 99%

Rules of the Road: Predicting Driving Behavior With a Convolutional Model of Semantic Interactions

Hong

Sapp

Philbin

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

249

166

View full text Add to dashboard Cite

We focus on the problem of predicting future states of entities in complex, real-world driving scenarios. Previous research has used low-level signals to predict short time horizons, and has not addressed how to leverage key assets relied upon heavily by industry self-driving systems:(1) large 3D perception efforts which provide highly accurate 3D states of agents with rich attributes, and (2) detailed and accurate semantic maps of the environment (lanes, traffic lights, crosswalks, etc). We present a unified representation which encodes such high-level semantic information in a spatial grid, allowing the use of deep convolutional models to fuse complex scene context. This enables learning entity-entity and entity-environment interactions with simple, feed-forward computations in each timestep within an overall temporal model of an agent's behavior. We propose different ways of modelling the future as a distribution over future states using standard supervised learning. We introduce a novel dataset providing industry-grade rich perception and semantic inputs, and empirically show we can effectively learn fundamentals of driving behavior.

show abstract

“…However, while theirs is learned in combination with the model, ours is a fixed deterministic mapping intended to be used as a problem and model independent metric. Rhinehart et al introduce two entropy terms as loss functions for trajectory prediction, encouraging predictions to be diverse whilst simultaneously precise [13]. While again very similar in motivation, this also is no metric.…”

Section: Related Workmentioning

confidence: 99%

Ambiguity in Sequential Data: Predicting Uncertain Futures With Recurrent Models

Berlati

Scheel

Stefano

et al. 2020

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Ambiguity is inherently present in many machine learning tasks, but especially for sequential models seldom accounted for, as most only output a single prediction. In this work we propose an extension of the Multiple Hypothesis Prediction (MHP) model to handle ambiguous predictions with sequential data, which is of special importance, as often multiple futures are equally likely. Our approach can be applied to the most common recurrent architectures and can be used with any loss function. Additionally, we introduce a novel metric for ambiguous problems, which is better suited to account for uncertainties and coincides with our intuitive understanding of correctness in the presence of multiple labels. We test our method on several experiments and across diverse tasks dealing with time series data, such as trajectory forecasting and maneuver prediction, achieving promising results.

show abstract

r2p2: A ReparameteRized Pushforward Policy for Diverse, Precise Generative Path Forecasting

Cited by 205 publications

References 32 publications

PRECOG: PREdiction Conditioned on Goals in Visual Multi-Agent Settings

PRECOG: PREdiction Conditioned on Goals in Visual Multi-Agent Settings

Rules of the Road: Predicting Driving Behavior With a Convolutional Model of Semantic Interactions

Ambiguity in Sequential Data: Predicting Uncertain Futures With Recurrent Models

Contact Info

Product

Resources

About