DLow: Diversifying Latent Flows for Diverse Human Motion Prediction

Yuan, Ye

doi:10.48550/arxiv.2003.08386

Cited by 5 publications

(13 citation statements)

References 72 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We design a diversity loss objective to specifically increase the safety and effectiveness of the downstream task of motion planning. This stands in contrast to previous works that simply promote diversity in euclidean space [35,36], which might promote diversity from actors that are not relevant, or actions that do not impact the SDV behavior. Towards this goal, we reward sets of diverse predictions that lead to greater diversity in the SDV's planned trajectory for each future scenario.…”

Section: Introductioncontrasting

confidence: 85%

“…This is especially important in motion forecasting as SDVs need to be able to anticipate rare behavior by other actors on the road in order to plan for safe responses to dangerous or unexpected driving. Recent work [35,36] has developed methods that encourage more diverse prediction for pretrained variational inference models. They train new encoders that output a fixed number of jointly diverse samples of latent variables.…”

Section: Related Workmentioning

confidence: 99%

“…Samples from this latent distribution are passed through the pretrained decoder and parameterize a distribution of diverse motion forecasts for the scene. In the work of [36], a diversity loss based on the L2-distance between motion forecast samples is used to increase diversity in the model. However, the framework is more flexible and we leverage it to introduce more prior knowledge at the output level.…”

Section: Related Workmentioning

confidence: 99%

“…. This paramaterization is similar to [36], with the difference that their latent dimension has fixed size and ours depends on the number of actors N in the scene. This is required in our setting since we are modeling the joint distribution over actors' trajectories, in contrast to their marginals [36].…”

Section: Diverse Samplermentioning

confidence: 99%

“…In this work, we demonstrate that by combining generative models with a diverse sampling function [35,36,27], we can leverage the wealth of large-scale datasets to learn high fidelity multi-agent behavioral models, achieve good coverage of future modes with a small set of samples, and remove any stochasticity in the autonomy stack. We design a diversity loss objective to specifically increase the safety and effectiveness of the downstream task of motion planning.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

LookOut: Diverse Multi-Future Prediction and Planning for Self-Driving

Cui¹,

Casas

Sadat

et al. 2021

Preprint

View full text Add to dashboard Cite

Self-driving vehicles need to anticipate a diverse set of future traffic scenarios in order to safely share the road with other traffic participants that may exhibit rare but dangerous driving. In this paper, we present LOOKOUT, an approach to jointly perceive the environment and predict a diverse set of futures from sensor data, estimate their probability, and optimize a contingency plan over these diverse future realizations. In particular, we learn a diverse joint distribution over multi-agent future trajectories in a traffic scene that allows us to cover a wide range of future modes with high sample efficiency while leveraging the expressive power of generative models. Unlike previous work in diverse motion forecasting, our diversity objective explicitly rewards sampling future scenarios that require distinct reactions from the self-driving vehicle for improved safety. Our contingency planner then finds comfortable trajectories that ensure safe reactions to a wide range of future scenarios. Through extensive evaluations, we show that our model demonstrates significantly more diverse and sample-efficient motion forecasting in a largescale self-driving dataset as well as safer and more comfortable motion plans in long-term closed-loop simulations than current state-of-the-art models.

show abstract

Section: Introductioncontrasting

confidence: 85%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Diverse Samplermentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

LookOut: Diverse Multi-Future Prediction and Planning for Self-Driving

Cui¹,

Casas

Sadat

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

Generative Hybrid Representations for Activity Forecasting With No-Regret Learning

Guan

Yuan

Rhinehart

2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

View full text Add to dashboard Cite

Automatically reasoning about future human behaviors is a difficult problem but has significant practical applications to assistive systems. Part of this difficulty stems from learning systems' inability to represent all kinds of behaviors. Some behaviors, such as motion, are best described with continuous representations, whereas others, such as picking up a cup, are best described with discrete representations. Furthermore, human behavior is generally not fixed: people can change their habits and routines. This suggests these systems must be able to learn and adapt continuously. In this work, we develop an efficient deep generative model to jointly forecast a person's future discrete actions and continuous motions. On a large-scale egocentric dataset, EPIC-KITCHENS, we observe our method generates high-quality and diverse samples while exhibiting better generalization than related generative models. Finally, we propose a variant to continually learn our model from streaming data, observe its practical effectiveness, and theoretically justify its learning efficiency.

show abstract