Understanding Pedestrian-Vehicle Interactions with Vehicle Mounted Vision: An LSTM Model and Empirical Analysis

Abstract

Pedestrians and vehicles often share the road in complex inner city traffic. This leads to interactions between the vehicle and pedestrians, with each affecting the other's motion. In order to create robust methods to reason about pedestrian behavior and to design interfaces of communication between self-driving cars and pedestrians we need to better understand such interactions. In this paper, we present a datadriven approach to implicitly model pedestrians' interactions with vehicles, to better predict pedestrian behavior. We propose a Long Short-Term Memory (LSTM) model that takes as input the past trajectories of the pedestrian and ego-vehicle, and pedestrian head orientation, and predicts the future positions of the pedestrian. Our experiments based on a real-world, inner city dataset captured with vehicle mounted cameras, show that the usage of such cues improve pedestrian prediction when compared to a baseline that purely uses the past trajectory of the pedestrian. * This work was done when Daniela A. Ridel was a visiting scholar at the Laboratory for Intelligent and Safe Automobiles

Scene Compliant Trajectory Forecast With Agent-Centric Spatio-Temporal Grids

“…Alternatively, some approaches use temporal convolutional networks for encoding sequences of past locations [12], [13], allowing for faster run-times. In addition to location co-ordinates, some approaches also incorporate auxiliary information such as the head pose of pedestrians [9], [14] while encoding past motion. Many approaches jointly model the past motion of multiple agents in the scene to capture interaction between agents [5], [15], [12], [10], [7], [11].…”

Prediction of Pedestrian Risky Level for Intelligent Vehicles

“…• Secondly, to further simplify the motion modeling process, the long short-term memory (LSTM) technique is selected to build a time-series neural network. Such a network can infer the movement pattern of a pedestrian from various data types [17][18][19][20][21]. This work utilizes this data-driven approach to learn the vehicle-perspective data and predict the relative trajectory of pedestrians.…”

Scene compliant spatio-temporal multi-modal multi-agent long-term trajectory forecasting

“…Alternatively, some approaches use temporal convolutional networks for encoding sequences of past locations (LEE et al, 2017;NIKHIL;MORRIS, 2018), allowing for faster run-times. In addition to location coordinates, some approaches also incorporate auxiliary information such as the head pose of pedestrians (HASAN et al, 2018;RIDEL et al, 2019) while encoding past motion.…”

“…This is most commonly done by sampling generative models such as Generative Adversarial Networks (GANs) (GUPTA et al, 2018;SADEGHIAN et al, 2019; AMIRIAN; HAYET; PETTRÉ, 2019), Variational Autoencoders (VAEs) (LEE et al, 2017) and invertible models (RHINEHART; KITANI; VERNAZA, 2018). Some approaches sample a stochastic policy obtained using imitation learning or inverse reinforcement learning (Li, 2019;TRIVEDI, 2019). Other approaches learn mixture models (CUI et al, 2019;Zyner;Worrall;Nebot, 2019;.…”

“…When looking at humans and predicting their behaviors inside cities a common pipeline is first detecting them in 2D/3D images, then tracking them among consecutive images (video), by assigning a unique identifier, and then finally predicting their future behavior. The behavior prediction task was tackled in the literature in many forms, by classifying among many possible motion patterns GAVRILA, 2013;KOEHLER et al, 2013;BONNIN et al, 2014;VÖLZ et al, 2015;HASHIMOTO et al, 2015b;KWAK;KO;NAM, 2017) by predicting one future trajectory (QUINTERO et al, 2015;GOLDHAMMER et al, 2015;FERGUSON et al, 2015;SCHULZ;STIEFELHAGEN, 2015a), or by predicting many possible trajectories (GUPTA et al, 2018;SADEGHIAN et al, 2019;AMIRIAN;HAYET;PETTRÉ, 2019;LEE et al, 2017;TRIVEDI, 2019;CUI et al, 2019;Zyner;Worrall;Nebot, 2019).…”