CoverNet: Multimodal Behavior Prediction using Trajectory Sets

Phan-Minh, Tung; Grigore, Elena Corina; Boulton, Freddy A.; Beijbom, Oscar; Wolff, Eric M.

doi:10.48550/arxiv.1911.10298

Cited by 12 publications

(27 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Jointly consistent multi-agent forecasting Most existing models output independent future distributions per object in a scene, e.g. [1,3,7,5,8,12,11,14,17,20,22,25,39]. This is encouraged by the popular metrics, which only measure quality on a per-object level, and by datasets that only require predicting one agent per scene.…”

Section: Related Workmentioning

confidence: 99%

Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset

Ettinger¹,

Cheng²,

Caine³

et al. 2021

Preprint

View full text Add to dashboard Cite

As autonomous driving systems mature, motion forecasting has received increasing attention as a critical requirement for planning. Of particular importance are interactive situations such as merges, unprotected turns, etc., where predicting individual object motion is not sufficient. Joint predictions of multiple objects are required for effective route planning. There has been a critical need for highquality motion data that is rich in both interactions and annotation to develop motion planning models. In this work, we introduce the most diverse interactive motion dataset to our knowledge, and provide specific labels for interacting objects suitable for developing joint prediction models. With over 100,000 scenes, each 20 seconds long at 10 Hz, our new dataset contains more than 570 hours of unique data over 1750 km of roadways. It was collected by mining for interesting interactions between vehicles, pedestrians, and cyclists across six cities within the United States. We use a high-accuracy 3D auto-labeling system to generate high quality 3D bounding boxes for each road agent, and provide corresponding high definition 3D maps for each scene. Furthermore, we introduce a new set of metrics that provides a comprehensive evaluation of both single agent and joint agent interaction motion forecasting models. Finally, we provide strong baseline models for individualagent prediction and joint-prediction. We hope that this new large-scale interactive motion dataset will provide new opportunities for advancing motion forecasting models.

show abstract

Section: Related Workmentioning

confidence: 99%

Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset

Ettinger¹,

Cheng²,

Caine³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Motion Forecasting: In an autonomous driving system, a common approach for path planning is to detect the actors in the environment, predict where they are going in a short time-frame, and plan a safe path conditioned on those predictions. In the motion forecasting prediction step, a common approach for actor modeling has been to independently predict the trajectory of each actor [23,8,3,6,16,22,39]. These predictions can be represented as closed-form gaussian distributions [8,3,6], a classification or energy over a discrete grid/graph/set structure [16,39,22,37], or trajectory samples of a stochastic model [23,13].…”

Section: Related Workmentioning

confidence: 99%

“…In the motion forecasting prediction step, a common approach for actor modeling has been to independently predict the trajectory of each actor [23,8,3,6,16,22,39]. These predictions can be represented as closed-form gaussian distributions [8,3,6], a classification or energy over a discrete grid/graph/set structure [16,39,22,37], or trajectory samples of a stochastic model [23,13]. One approach to tractably model scene-consistent predictions is to stochastically sample one possible future scenario at a time, by sampling latent variables that encode the joint scene dynamics, and then decode the future trajectories.…”

Section: Related Workmentioning

confidence: 99%

“…sion making of the SDV. Most previous works in motion forecasting predict the marginal distribution of each actor's future trajectories [8,7,22,6,3,37] in a traffic scene. However, the marginal distribution does not reflect the future interactions among agents despite our knowledge that the path one agent may take will affect the distribution of paths of surrounding actors.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

LookOut: Diverse Multi-Future Prediction and Planning for Self-Driving

Cui¹,

Casas

Sadat

et al. 2021

Preprint

View full text Add to dashboard Cite

Self-driving vehicles need to anticipate a diverse set of future traffic scenarios in order to safely share the road with other traffic participants that may exhibit rare but dangerous driving. In this paper, we present LOOKOUT, an approach to jointly perceive the environment and predict a diverse set of futures from sensor data, estimate their probability, and optimize a contingency plan over these diverse future realizations. In particular, we learn a diverse joint distribution over multi-agent future trajectories in a traffic scene that allows us to cover a wide range of future modes with high sample efficiency while leveraging the expressive power of generative models. Unlike previous work in diverse motion forecasting, our diversity objective explicitly rewards sampling future scenarios that require distinct reactions from the self-driving vehicle for improved safety. Our contingency planner then finds comfortable trajectories that ensure safe reactions to a wide range of future scenarios. Through extensive evaluations, we show that our model demonstrates significantly more diverse and sample-efficient motion forecasting in a largescale self-driving dataset as well as safer and more comfortable motion plans in long-term closed-loop simulations than current state-of-the-art models.

show abstract

“…We thus use retrieval from a largescale dataset of real trajectories. This approach provides a large set of trajectories from expert demonstrations while avoiding random sampling or arbitrary choices of acceleration/steering profiles [36,55]. We create a dataset of expert demonstrations by binning based on the SDV initial state, clustering the trajectories of each bin, and using the cluster prototypes for efficiency.…”

Section: Trajectory Samplingmentioning

confidence: 99%

MP3: A Unified Model to Map, Perceive, Predict and Plan

Casas¹,

Sadat²,

Urtasun³

2021

Preprint

View full text Add to dashboard Cite

High-definition maps (HD maps) are a key component of most modern self-driving systems due to their valuable semantic and geometric information. Unfortunately, building HD maps has proven hard to scale due to their cost as well as the requirements they impose in the localization system that has to work everywhere with centimeter-level accuracy. Being able to drive without an HD map would be very beneficial to scale self-driving solutions as well as to increase the failure tolerance of existing ones (e.g., if localization fails or the map is not up-to-date). Towards this goal, we propose MP3, an end-to-end approach to mapless 1 driving where the input is raw sensor data and a high-level command (e.g., turn left at the intersection). MP3 predicts intermediate representations in the form of an online map and the current and future state of dynamic agents, and exploits them in a novel neural motion planner to make interpretable decisions taking into account uncertainty. We show that our approach is significantly safer, more comfortable, and can follow commands better than the baselines in challenging long-term closed-loop simulations, as well as when compared to an expert driver in a large-scale real-world dataset.

show abstract

CoverNet: Multimodal Behavior Prediction using Trajectory Sets

Cited by 12 publications

References 27 publications

Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset

Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset

LookOut: Diverse Multi-Future Prediction and Planning for Self-Driving

MP3: A Unified Model to Map, Perceive, Predict and Plan

Contact Info

Product

Resources

About