Multi-Modal Pedestrian Trajectory Prediction for Edge Agents Based on Spatial-Temporal Graph

Abstract

Edge agents, represented by socially-aware robots and autonomous vehicles, have gradually been integrated into human society. The safety navigation system in interactive scenes is of great importance to them. The key of this system is that the edge agent has the ability to predict the pedestrian trajectory in the dynamic scene, so as to avoid collision. However, predicting pedestrian trajectories in dynamic scenes is not an easy task, because it is necessary to comprehensively consider the spatial-temporal structure of human-environment interaction, visual attention, and the multi-modal behavior of human walking. In this paper, a scalable spatial-temporal graph generation adversarial network architecture (STG-GAN) is introduced, which can comprehensively consider the influence of human-environment interaction and generate a reasonable multi-modal prediction trajectory. First, we use LSTM nodes to flexibly transform the spatial-temporal graph of human-environment interactions into feed-forward differentiable feature coding, and innovatively propose the global node to integrate scene context information. Then, we capture the relative importance of global interactions on pedestrian trajectories through scaled dot product attention, and use recurrent sequence modeling and generative adversarial network architecture for common training, so as to generate reasonable pedestrian future trajectory distributions based on rich mixed features. Experiments on public data sets show that STG-GAN is superior to previous work in terms of accuracy, reasoning speed and rationality of trajectory prediction.

Nation-wide human mobility prediction based on graph neural networks

“…The proposal was successfully tested by using location data extracted from a Location-Based Service (LBS) in Beijing. Moreover, in a fine-grained spatial scale, an LSTM network is composed by Zou et al [27] to predict the collision-free trajectories of a set of pedestrians. Another example is an attention mechanism along with a bidirectional LSTM as stated by Zhao et al [29], to develop a user destination prediction based on LBS data.…”

Review of pedestrian trajectory prediction methods

“…However, S-GAN is not only simple in modeling pedestrian interaction, but also does not make full use of the deep interaction information of pedestrians. To this end, the subsequent methods explore the influencing factors of pedestrian trajectory by using attention mechanism [13,[49][50][51], increasing scene interaction [13,[52][53][54], feasibility constraints [46], etc. S-GAN and SoPhie are single behavior patterns with high variance, which are limited by social behavior and cannot learn the real multimodal distribution of pedestrians.…”

“…(1) Trajectory prediction based on space-time graph. Based on the wide application of graph convolutional networks in behavior recognition [58], traffic prediction [59], demand prediction [60], etc., many studies have tried to apply spatiotemporal graphs [61,52,[60][61][62][63][64][65] to pedestrian trajectory prediction tasks and achieved good prediction performance.…”

Trajectory prediction for autonomous driving based on multiscale spatial‐temporal graph

“…Therefore, some latest works attempted to utilize graph structure to represent and model the interactions that consider traffic agents as instance nodes and relationships as edges [12][13][14][15]. Furthermore, motivated by the fact that a traffic agent will be careful of both spatial and temporal interactions with surrounding agents to avoid future collisions, recently, various spatial-temporal graph models have been explored to improve prediction accuracy [15][16][17].…”