Multiple Trajectory Prediction with Deep Temporal and Spatial Convolutional Neural Networks

Abstract

Automated vehicles need to not only perceive their environment, but also predict the possible future behavior of all detected traffic participants in order to safely navigate in complex scenarios and avoid critical situations, ranging from merging on highways to crossing urban intersections. Due to the availability of datasets with large numbers of recorded trajectories of traffic participants, deep learning based approaches can be used to model the behavior of road users. This paper proposes a convolutional network that operates on rasterized actor-centric images which encode the static and dynamic actor-environment. We predict multiple possible future trajectories for each traffic actor, which include position, velocity, acceleration, orientation, yaw rate and position uncertainty estimates. To make better use of the past movement of the actor, we propose to employ temporal convolutional networks (TCNs) and rely on uncertainties estimated from the previous object tracking stage. We evaluate our approach on the public "Argoverse Motion Forecasting" dataset, on which it won the first prize at the Argoverse Motion Forecasting Challenge, as presented on the NeurIPS 2019 workshop on "Machine Learning for Autonomous Driving".

DeepSIL: A Software-in-the-Loop Framework for Evaluating Motion Planning Schemes Using Multiple Trajectory Prediction Networks

“…For the prediction of other road users, recently, several deep-learning methods have been proposed, e.g. [8], [16]. Nevertheless, while these neural networks do produce highly realistic trajectories, they do not offer the possibility to manipulate the trajectories, i.e.…”

“…In our work, we use the network from our previous work [8]. It was trained on the Argoverse dataset [9], and produces state-of-the-art results.…”

“…It was trained on the Argoverse dataset [9], and produces state-of-the-art results. In contrast to [8], we do not feed state uncertainties to the neural network during training, since the data from the simulator comes with no uncertainty. Also, we use motion history subsets of random size during training, to ensure that the model can also predict accurately if it does not have a full two seconds of history available.…”

“…We use the Lanelet2 API to load maps and traffic rules, and to construct reference paths from decision lists in the simulator. The neural network from [8] is implemented in PyTorch, and is executed by ONNX runtime with TensorRT acceleration. The synthetic images required for the MTP network are rendered directly on the GPU, where also the model inference takes place, which mainly consists of a convolutional neural network (CNN).…”

“…In this work, we propose DeepSIL, a software-in-the-loop (SIL) framework that combines a deep neural network for multiple trajectory prediction (MTP), e.g. [8], with a classical rule-based logic layer and a kinematic bicycle model. By using a kinematic model of the vehicles, a physically realistic motion of the simulated vehicles is ensured.…”