Self-Supervised Simultaneous Multi-Step Prediction of Road Dynamics and Cost Map

Abstract

While supervised learning is widely used for perception modules in conventional autonomous driving solutions, scalability is hindered by the huge amount of data labeling needed. In contrast, while end-to-end architectures do not require labeled data and are potentially more scalable, interpretability is sacrificed. We introduce a novel architecture that is trained in a fully self-supervised fashion for simultaneous multi-step prediction of space-time cost map and road dynamics. Our solution replaces the manually designed cost function for motion planning with a learned high dimensional cost map that is naturally interpretable and allows diverse contextual information to be integrated without manual data labeling. Experiments on real world driving data show that our solution leads to lower number of collisions and road violations in long planning horizons in comparison to baselines, demonstrating the feasibility of fully self-supervised prediction without sacrificing scalability.

LatentFormer: Multi-Agent Transformer-Based Interaction Modeling and Trajectory Prediction

“…Trajectory prediction is of vital importance to many artificial intelligent applications. There is a large body of works on this topic designed to predict behavior of pedestrians [21,28,37,45,50] and vehicles [1,10,11,13,19,36].…”

Deep Occupancy-Predictive Representations for Autonomous Driving

“…With the objective of encoding traffic scenes similar to ours (albeit not in the context of motion planning), encoder architectures for learning representations of occupancy maps have been proposed [50]- [52]. Using graphical or otherwise spatially-aware encoders similar to ours, recent works such as [53]- [57] predict occupancy grids [58] as an intermediate learning target for guiding the training of neural motion planners. However, these approaches do not provide global, low-dimensional representations appropriate for decoupled RL agents.…”

MotionSC: Data Set and Network for Real-Time Semantic Mapping in Dynamic Environments

“…Semantic scene completion is a similar task to semantic mapping, however it is generally defined using sensor data from only a single frame. While there exist some deep learning 3D mapping methods, it is not a common task due to the lack of accurate outdoor dynamic data to supervise and quantify performance on [17,[45][46][47][48]. SSC is currently a difficult task with minimal generalizability to real life due to the lack of accurately labeled dynamic scenes, as discussed in the previous section.…”

“…Dynamic occupancy maps construct binary labels for cells indicating free or occupied, and extend their domain to scenes with dynamic actors by incorporating scene dynamics [13][14][15]. While learning-based approaches have been attempted in 2D [16,17], most 3D maps rely on feature engineering which can decrease performance and efficiency.…”