Forecasting Interactive Dynamics of Pedestrians with Fictitious Play

Abstract

We develop predictive models of pedestrian dynamics by encoding the coupled nature of multi-pedestrian interaction using game theory and deep learning-based visual analysis to estimate person-specific behavior parameters. We focus on predictive models since they are important for developing interactive autonomous systems (e.g., autonomous cars, home robots, smart homes) that can understand different human behavior and pre-emptively respond to future human actions. Building predictive models for multi-pedestrian interactions however, is very challenging due to two reasons: (1) the dynamics of interaction are complex interdependent processes, where the decision of one person can affect others; and (2) dynamics are variable, where each person may behave differently (e.g., an older person may walk slowly while the younger person may walk faster). We address these challenges by utilizing concepts from game theory to model the intertwined decision making process of multiple pedestrians and use visual classifiers to learn a mapping from pedestrian appearance to behavior parameters. We evaluate our proposed model on several public multiple pedestrian interaction video datasets. Results show that our strategic planning model predicts and explains human interactions 25% better when compared to a state-of-the-art activity forecasting method.

Looking to Relations for Future Trajectory Forecast

“…Such technologies require advanced decision making and motion planning systems that rely on estimates of the future position of road users in order to realize safe and effective mitigation and navigation strategies. Related research [46,1,36,23,37,12,13,43,45,32,33,47] has attempted to predict future trajectories by focusing on social conventions, environmental factors, or pose and motion constraints. They have shown to be more effective when the prediction model learns to extract these features by considering human-human (i.e., between road agents) or human-space (i.e., between a road agent and environment) interactions.…”

Forecasting People Trajectories and Head Poses by Jointly Reasoning on Tracklets and Vislets

“…Helbing and Molnar [38] have considered for the first time the effect of other pedestrians to the behavior of an individual. The pioneering idea has been further developed by [52], [55] and [60], who have respectively introduced a data-driven, continuous and game theoretical model. Notably, these approaches successfully employed the essential cues for track prediction, such as the human-human interaction and people intended destination.…”

Survey on Vision-Based Path Prediction

“…Stacked hierarchical labeling [7] Superpixel-based MRF [8] Fully convolutional networks [9,10] Cost Bag of visual words Spatial matching network [11] Global scene feature Pre-trained AlexNet [12] Siamese network [13] Location HOG + SVM detector [14] Target Direction Bayesian orientation estimation [15] Orientation network [11] Attribute AlexNet-based multi-task learning [16] Feature vector Mid-level patch features [17] 2 Feature extraction from a video…”

“…Aspects of such avoidance -when and where pedestrians start to avoid others -are different for pedestrians of different age and gender; e.g., a younger person walks faster and responds more rapidly to others than senior people. Wei et al [16] used AlexNet to estimate the orientation, age, and gender of pedestrians as multi-task learning. Estimated attributes are used in deciding the walking speed of pedestrians.…”

“…They predicted paths even in scenes with occlusions by estimating regions behind occluding objects, such as walls and obstacles. Su et al [24] extended this work to the prediction of multiple basketball players Datasets and results of prediction, taken and modified from [16,29,31,38,18,28,25,21,23,34].…”