Social and Scene-Aware Trajectory Prediction in Crowded Spaces

Abstract

Mimicking human ability to forecast future positions or interpret complex interactions in urban scenarios, such as streets, shopping malls or squares, is essential to develop socially compliant robots or self-driving cars. Autonomous systems may gain advantage on anticipating human motion to avoid collisions or to naturally behave alongside people. To foresee plausible trajectories, we construct an LSTM (long short-term memory)-based model considering three fundamental factors: people interactions, past observations in terms of previously crossed areas and semantics of surrounding space. Our model encompasses several pooling mechanisms to join the above elements defining multiple tensors, namely social, navigation and semantic tensors. The network is tested in unstructured environments where complex paths emerge according to both internal (intentions) and external (other people, not accessible areas) motivations. As demonstrated, modeling paths unaware of social interactions or context information, is insufficient to correctly predict future positions. Experimental results corroborate the effectiveness of the proposed framework in comparison to LSTM-based models for human path prediction.

Human Trajectory Forecasting in Crowds: A Deep Learning Perspective

“…RNNs incorporating social interactions allow anticipating interactions that can occur in a more distant future. The social pooling module has been extended to incorporate physical space context [41]- [47] and various other designs of NN-based interaction module have been proposed [48]- [61]. Pfieffer et al [48] proposed an angular pooling grid for efficient computation.…”

“…Neighbour Input State: Consider an N o × N o grid around the primary pedestrian, where each cell contains information about neighbours located in that corresponding position. Existing designs provides the information of the neighbours in two main forms: (a) Occupancy Pooling [15], [44] where each cell in the grid indicates the presence of a neighbour (see Fig 3a) (b) Social Pooling [15], [42]- [44], [46], [47], [51] where each cell contains the entire past history of the neighbour, represented by, e.g., the LSTM hidden state of the neighbours (see Fig 3c). The obtained grid is flattened and subsequently embedded using an MLP to get the interaction vector p t i .…”

Multiple Trajectory Prediction of Moving Agents With Memory Augmented Networks

“…Trajectory Prediction Significant effort has been made in the past years regarding trajectory prediction. Several researchers have focused on trajectories of pedestrians [6], [7], [46], [47], [48], either regarded as individuals or crowds, also exploiting social behaviors and interactivity between individuals [6], [7], [46], [47], [49], [50].…”

Imitative Non-Autoregressive Modeling for Trajectory Forecasting and Imputation

“…Trajectory Prediction and Imputation. How to predict future motion trajectory [2,4,10,17,18,21,24,27,30,42,48,50,51] and impute missing value [6,13,29,46,47] in sequences are very important yet challenging tasks. For instance, Zheng et al [51] proposed a deep hierarchical policy model and Felsen et al [10] utilized conditional variational autoencoders to predict fine-grained multi-agent motions.…”