SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints

Sadeghian, Amir; Kosaraju, Vineet; Sadeghian, Ali; Hirose, Noriaki; Rezatofighi, S. Hamid; Savarese, Silvio

doi:10.48550/arxiv.1806.01482

Cited by 39 publications

(142 citation statements)

References 0 publications

Supporting

Mentioning

125

Contrasting

Order By: Relevance

“…S-GAN [24] improves on S-LSTM by introducing generative adversarial network to it. SoPhie [25] leverages a social attention module which learns the interactions between agents, and a physical attention module which learns the physical constraints in the scene, to predict trajectories. SRLSTM [26] integrates a state refinement module into LSTM to achieve more accurate joint trajectory predictions.…”

Section: B Deep Learning Approaches For Motion Predictionmentioning

confidence: 99%

GAMMA: A General Agent Motion Model for Autonomous Driving

Luo¹,

Cai²,

Lee³

et al. 2019

Preprint

View full text Add to dashboard Cite

Autonomous driving in mixed traffic requires reliable motion prediction of nearby traffic agents such as pedestrians, bicycles, cars, buses, etc.. This prediction problem is extremely challenging because of the diverse dynamics and geometry of traffic agents, complex road conditions, and intensive interactions among the agents. In this paper, we proposed GAMMA, a general agent motion prediction model for autonomous driving, that can predict the motion of heterogeneous traffic agents with different kinematics, geometry, human agents' inner states, etc.. GAMMA formalizes motion prediction as geometric optimization in the velocity space, and integrates physical constraints and human inner states into this unified framework. Our results show that GAMMA outperforms state-of-the-art approaches significantly on diverse real-world datasets.

show abstract

Section: B Deep Learning Approaches For Motion Predictionmentioning

confidence: 99%

GAMMA: A General Agent Motion Model for Autonomous Driving

Luo¹,

Cai²,

Lee³

et al. 2019

Preprint

View full text Add to dashboard Cite

show abstract

“…In the first ablation, we use normal birdview images up to time T obs and then blank images subsequently, both at training and test time. This removes any feedback and roughly corresponds to models that predict the entire future trajectory in a single step based on past observations [5], [6], [7], [8], [9], albeit with a suboptimal neural architecture. In the second ablation, we perform teacher forcing at training time, fixing the agent's state and the birdview image to the ground truth values at each time step.…”

Section: B Ablations On Future Birdview Imagesmentioning

confidence: 99%

“…The most popular alternative to VAEs are generative adversarial networks (GANs) [29], which have been been applied to trajectory prediction in papers such as Social GAN [7], SoPhie [6], and [8]. These approaches differ in how they construct the loss functions and how they encode the information about the past, but in all of them predictions for different agents are generated independently, by decoding a random variable with an RNN.…”

Section: Related Workmentioning

confidence: 99%

“…Most of the existing approaches to trajectory prediction [5], [6], [7], [8], [9] treat it as a regression problem, encoding past states of the world to produce a distribution over trajectories for each agent in a relatively black-box manner, without accounting for future interactions. Several notable papers [10], [11], [12], [13] explicitly model trajectory generation as a sequential decision-making process, allowing the agents to interact in the future.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Imagining The Road Ahead: Multi-Agent Trajectory Prediction via Differentiable Simulation

Ścibior¹,

Lioutas²,

Reda³

et al. 2021

Preprint

View full text Add to dashboard Cite

We develop a deep generative model built on a fully differentiable simulator for multi-agent trajectory prediction. Agents are modeled with conditional recurrent variational neural networks (CVRNNs), which take as input an ego-centric birdview image representing the current state of the world and output an action, consisting of steering and acceleration, which is used to derive the subsequent agent state using a kinematic bicycle model. The full simulation state is then differentiably rendered for each agent, initiating the next time step. We achieve state-of-the-art results on the INTERACTION dataset, using standard neural architectures and a standard variational training objective, producing realistic multi-modal predictions without any ad-hoc diversity-inducing losses. We conduct ablation studies to examine individual components of the simulator, finding that both the kinematic bicycle model and the continuous feedback from the birdview image are crucial for achieving this level of performance. We name our model ITRA, for "Imagining the Road Ahead".

show abstract

“…There are many works trying to cope with the above challenges in different aspects: single agent-to-agent interaction [1,13,31], single and group interaction [3,6,30,39], agent-to-environment interaction [4,20], considering both interaction and environmental factors but only for homogeneous agents (e.g., pedestrians) [37][38][39]. Recently, more and more works address the problems of trajectory prediction by generating multiple paths [13,23,32,41] and generalize the task for mixed traffic [5,7,8,30,34]. However, it lacks work that comprehensively tackles the aforementioned challenges within one framework for mixed traffic multi-path trajectory prediction.…”

Section: Introductionmentioning

confidence: 99%

MCENET: Multi-Context Encoder Network for Homogeneous Agent Trajectory Prediction in Mixed Traffic

Cheng

Liao

Yang

et al. 2020

Preprint

View full text Add to dashboard Cite

Trajectory prediction in urban mixed-traffic zones (a.k.a. shared spaces) is critical for many intelligent transportation systems, such as intent detection for autonomous driving. However, there are many challenges to predict the trajectories of heterogeneous road agents (pedestrians, cyclists and vehicles) at a microscopical level. For example, an agent might be able to choose multiple plausible paths in complex interactions with other agents in varying environments. To this end, we propose an approach named Multi-Context Encoder Network (MCENET) that is trained by encoding both past and future scene context, interaction context and motion information to capture the patterns and variations of the future trajectories using a set of stochastic latent variables. In inference time, we combine the past context and motion information of the target agent with samplings of the latent variables to predict multiple realistic trajectories in the feature. Through experiments on several datasets of varying scenes, our method outperforms some of the recent state-of-the-art methods for mixed traffic trajectory prediction by a large margin and more robust in a very challenging environment. The impact of each context is justified via ablation studies.

show abstract

SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints

Cited by 39 publications

References 0 publications

GAMMA: A General Agent Motion Model for Autonomous Driving

GAMMA: A General Agent Motion Model for Autonomous Driving

Imagining The Road Ahead: Multi-Agent Trajectory Prediction via Differentiable Simulation

MCENET: Multi-Context Encoder Network for Homogeneous Agent Trajectory Prediction in Mixed Traffic

Contact Info

Product

Resources

About