Soft + Hardwired attention: An LSTM framework for human trajectory prediction and abnormal event detection

Abstract

As humans we possess an intuitive ability for navigation which we master through years of practice; however existing approaches to model this trait for diverse tasks including monitoring pedestrian flow and detecting abnormal events have been limited by using a variety of hand-crafted features. Recent research in the area of deeplearning has demonstrated the power of learning features directly from the data; and related research in recurrent neural networks has shown exemplary results in sequenceto-sequence problems such as neural machine translation and neural image caption generation. Motivated by these approaches, we propose a novel method to predict the future motion of a pedestrian given a short history of their, and their neighbours, past behaviour. The novelty of the proposed method is the combined attention model which utilises both "soft attention" as well as "hard-wired" attention in order to map the trajectory information from the local neighbourhood to the future positions of the pedestrian of interest. We illustrate how a simple approximation of attention weights (i.e hard-wired) can be merged together with soft attention weights in order to make our model applicable for challenging real world scenarios with hundreds of neighbours. The navigational capability of the proposed method is tested on two challenging publicly available surveillance databases where our model outperforms the currentstate-of-the-art methods. Additionally, we illustrate how the proposed architecture can be directly applied for the task of abnormal event detection without handcrafting the features.

PRECOG: PREdiction Conditioned on Goals in Visual Multi-Agent Settings

“…While this does not test our model in a full control scenario, it does allow us to evaluate whether conditioning on the goal provides more accurate and higher-confidence predictions. We use our model's multi-agent prior (5) in the stochastic latent multi-agent planning objective (10), and define the goal-likelihood p(G|S, φ) = N (S r T ; S * r T , 0.1·I), i.e. a normal distribution at the controlled agent's last true future position, S * r T .…”

GD-GAN: Generative Adversarial Networks for Trajectory Prediction and Group Detection in Crowds

“…Social Force models [17,34], which rely on the attractive and repulsive forces between pedestrians to model their future behaviour, have been extensively applied for modelling human navigational behaviour. However with the dawn of deep learning, these methods have been replaced as they have been shown to ill represent the structure of human decision making [7,8,15]. One of the most popular deep learning methods is the social LSTM [1] model which represents the pedestrians in the local neighbourhood using LSTMs and then generates their future trajectory by systematically pooling the relavant information.…”

“…To this end we propose a deep learning algorithm which automatically learns these group attributes. We take inspiration from the trajectory modelling approaches of [8] and [11], where the approaches capture contextual information from the local neighbourhood. We further augment this approach with a Generative Adversarial Network (GAN) [10,15,28] learning pipeline where we learn a custom, task specific loss function which is specifically tailored for future trajectory prediction, learning to imitate complex human behaviours.…”

StarNet: Pedestrian Trajectory Prediction using Deep Neural Network in Star Topology

“…Data-driven methods remove the requirement of handcrafted features, and greatly improve the ability to predict pedestrian trajectories. Some attempts [11], [13], [14], [26], [27] receive pedestrian positions and predict determined trajectories. Social LSTM [11] devises social pooling to deal with interpersonal interactions.…”