Transformer Networks for Trajectory Forecasting

Giuliari, Francesco; Hasan, Irtiza; Cristani, Marco; Galasso, Fabio

doi:10.1109/icpr48806.2021.9412190

Cited by 231 publications

(162 citation statements)

References 28 publications

Supporting

Mentioning

153

Contrasting

Order By: Relevance

“…Additionally, a few first transformer models have entered the field of scene prediction, currently with the focus more on human traffic prediction [120,121] or long-term traffic flow (time interval on several minutes) [122]. As in Natural Language Processing (NLP) transformers partially begin to challenge RNN-based sequence-to-sequence models in (human) trajectory too, demonstrating the importance of the attention module in those tasks.…”

Section: Gnns Attention and New Use Casesmentioning

confidence: 99%

A Review on Scene Prediction for Automated Driving

Stockem

Krüger

Stolpe

et al. 2022

Physics

View full text Add to dashboard Cite

Towards the aim of mastering level 5, a fully automated vehicle needs to be equipped with sensors for a 360∘ surround perception of the environment. In addition to this, it is required to anticipate plausible evolutions of the traffic scene such that it is possible to act in time, not just to react in case of emergencies. This way, a safe and smooth driving experience can be guaranteed. The complex spatio-temporal dependencies and high dynamics are some of the biggest challenges for scene prediction. The subtile indications of other drivers’ intentions, which are often intuitively clear to the human driver, require data-driven models such as deep learning techniques. When dealing with uncertainties and making decisions based on noisy or sparse data, deep learning models also show a very robust performance. In this survey, a detailed overview of scene prediction models is presented with a historical approach. A quantitative comparison of the model results reveals the dominance of deep learning methods in current state-of-the-art research in this area, leading to a competition on the cm scale. Moreover, it also shows the problem of inter-model comparison, as many publications do not use standardized test sets. However, it is questionable if such improvements on the cm scale are actually necessary. More effort should be spent in trying to understand varying model performances, identifying if the difference is in the datasets (many simple situations versus many corner cases) or actually an issue of the model itself.

show abstract

Section: Gnns Attention and New Use Casesmentioning

confidence: 99%

A Review on Scene Prediction for Automated Driving

Stockem

Krüger

Stolpe

et al. 2022

Physics

View full text Add to dashboard Cite

show abstract

“…The Trajectron [21] combines elements from CVAE, LSTM, and dynamic Spatio-temporal graphical structures to produce multimodal trajectories. Recently, a Transformer model was proposed by [15] to predict the future trajectories of the pedestrians conditioning on the previous displacement of each pedestrian in the scene. The transformer used has the same architecture as the Vanilla Transformer proposed in [11].…”

Section: Related Workmentioning

confidence: 99%

“…Transformers have first revolutionized the natural language processing problems by outperforming all the previously proposed solutions [12], [13]. However, it was only until recently that were proved equally efficient for non-NLP problems [14], [15], [16].…”

Section: Introductionmentioning

confidence: 99%

Is attention to bounding boxes all you need for pedestrian action prediction?

Achaji¹,

Moreau²,

Fouqueray³

et al. 2021

Preprint

View full text Add to dashboard Cite

The human driver is no longer the only one concerned with the complexity of the driving scenarios. Autonomous vehicles (AV) are similarly becoming involved in the process. Nowadays, the development of AV in urban places underpins essential safety concerns for vulnerable road users (VRUs) such as pedestrians. Therefore, to make the roads safer, it is critical to classify and predict their future behavior. In this paper, we present a framework based on multiple variations of the Transformer models to reason attentively about the dynamic evolution of the pedestrians' past trajectory and predict its future actions of crossing or not crossing the street. We proved that using only bounding-boxes as input to our model can outperform the previous state-of-the-art models and reach a prediction accuracy of 91% and an F1-score of 0.83 on the PIE dataset up to two seconds ahead in the future. In addition, we introduced a large-size simulated dataset (CP2A) using CARLA for action prediction. Our model has similarly reached high accuracy (91 %) and F1-score (0.91) on this dataset. Interestingly, we showed that pre-training our Transformer model on the simulated dataset and then fine-tuning it on the real dataset can be very effective for the action prediction task.

show abstract

“…However, taking into account interactions between pedestrians requires to predict the coordinates one step at a time, so RNNs are generally preferred. Most of past years research focused on improving the interaction module, with only limited new methods since Social-LSTM [1], or in developing approaches that take inspiration in popular frameworks such as Transformers [14] or contrastive learning [15] in order to deter the model from predicting colliding or too uncomfortable trajectories. However, little work has been published on the influence of the encoder and thus on the importance of past coordinates, even if it would be easily applicable on all models that use this pipeline.…”

Section: A Encoder-interaction-decoder Pipelinementioning

confidence: 99%

“…Convolutional Neural Networks are faster than RNN-based methods due to parallelization, but the performances are significantly lower [18]. Some authors have explored the popular Transformers architecture, but the results are inferior to those of RNNs with state-of-the-art social interaction modules [14]. Research has also been conducted on applying Inverse Reinforcement Learning (IRL) to the pedestrian trajectory prediction problem [19], even though retrieving the pedestrian cost function requires much more computation than learning a predictor.…”

Section: A Encoder-interaction-decoder Pipelinementioning

confidence: 99%

Asymmetrical Bi-RNN for pedestrian trajectory encoding

Rozenberg,

Gesnouin,

Moutarde

2021

Preprint

View full text Add to dashboard Cite

Pedestrian motion behavior involves a combination of individual goals and social interactions with other agents. In this article, we present an asymmetrical bidirectional recurrent neural network architecture called U-RNN to encode pedestrian trajectories and evaluate its relevance to replace LSTMs for various forecasting models. Experimental results on the Trajnet++ benchmark show that the U-LSTM variant yields better results regarding every available metrics (ADE, FDE, Collision rate) than common trajectory encoders for a variety of approaches and interaction modules, suggesting that the proposed approach is a viable alternative to the de facto sequence encoding RNNs. Our implementation of the asymmetrical Bi-RNNs for the Trajnet++ benchmark is available at: github.com/JosephGesnouin/Asymmetrical-Bi-RNNs-toencode-pedestrian-trajectories

show abstract

Transformer Networks for Trajectory Forecasting

Cited by 231 publications

References 28 publications

A Review on Scene Prediction for Automated Driving

A Review on Scene Prediction for Automated Driving

Is attention to bounding boxes all you need for pedestrian action prediction?

Asymmetrical Bi-RNN for pedestrian trajectory encoding

Contact Info

Product

Resources

About