How many Observations are Enough? Knowledge Distillation for Trajectory Forecasting

Abstract

Accurate prediction of future human positions is an essential task for modern video-surveillance systems. Current state-of-the-art models usually rely on a "history" of past tracked locations (e.g., 3 to 5 seconds) to predict a plausible sequence of future locations (e.g., up to the next 5 seconds). We feel that this common schema neglects critical traits of realistic applications: as the collection of input trajectories involves machine perception (i.e., detection and tracking), incorrect detection and fragmentation errors may accumulate in crowded scenes, leading to tracking drifts. On this account, the model would be fed with corrupted and noisy input data, thus fatally affecting its prediction performance.In this regard, we focus on delivering accurate predictions when only few input observations are used, thus potentially lowering the risks associated with automatic perception. To this end, we conceive a novel distillation strategy that allows a knowledge transfer from a teacher network to a student one, the latter fed with fewer observations (just two ones). We show that a properly defined teacher supervision allows a student network to perform comparably to state-of-the-art approaches that demand more observations. Besides, extensive experiments on common trajectory forecasting datasets highlight that our student network better generalizes to unseen scenarios.

View Vertically: A Hierarchical Network for Trajectory Prediction via Fourier Spectrums

“…Interestingly, Becker et al [51] conduct a series of experiments to show that the latest two steps contribute to the prediction for a surprising 88.3%. Similarly, Monti et al [31] find that while earlier steps exert an (albeit small) influence, subsequent states provide a higher contribution. In short, the destinations may play significant roles when forecasting.…”

“…Alahi et al [17] treat this task as sequence generation and employ LSTMs to model and predict pedestrians' positions in the next time step recurrently. With the success of Transformers [61] in sequence processing such as natural language processing, researchers like [28], [29], [31], [44], [62] have designed different Transformers to obtain better trajectory representations. In addition, several factors, such as social/scene interactions [22], [23], [63], [64], [65], [66] and stochastic trajectory prediction [18], [19], [20], [32], [67], [68], [69], have been widely investigated.…”

“…Numerous researchers have explored creative ways to model and forecast trajectories by considering these interactions. Neural networks like Long-Short Term Memory Networks (LSTMs) [21], [22], [23], Graph Neural Networks (GNNs) [24], [25], [26], [27], and Transformers [28], [29], [30], [31] are employed to encode agents' trajectories. Besides, agents' motion preferences [2], [32], [33] and their goal distributions [34], [35], [36] have been investigated.…”

ENIGMA: Egocentric Navigator for Industrial Guidance, Monitoring and Anticipation

“…Indeed, the authors of [10] proposed a generic framework to anticipate future events in team sports videos directly from visual inputs. The anticipation problem has been explored even predicting the future location of users which allows to build an advanced surveillance systems able to predict people's activities [23,29] or for autonomous vehicles to understand pedestrian intents to avoid accidents [27,28]. In particular, the authors of [27] tackled the problem to jointly predicting the future spatial position and the body keypoints of pedestrians to have a deeper understanding of pedestrians behavior.…”

CRAT-Pred: Vehicle Trajectory Prediction with Crystal Graph Convolutional Neural Networks and Multi-Head Self-Attention

“…Neitz et al [30] apply principles of knowledge distillation to the field of motion prediction, allowing to combine advantages of model-based and modelfree prediction techniques. Monti et al [31] use a teacher and student of the same architecture to obtain a model that is able to predict human motion with a limited amount of observed timesteps. A higher amount of input observations are used for the teacher than for the student.…”