Dynamic Occupancy Grid Prediction for Urban Autonomous Driving: A Deep Learning Approach with Fully Automatic Labeling

Abstract

Long-term situation prediction plays a crucial role in the development of intelligent vehicles. A major challenge still to overcome is the prediction of complex downtown scenarios with multiple road users, e.g., pedestrians, bikes, and motor vehicles, interacting with each other. This contribution tackles this challenge by combining a Bayesian filtering technique for environment representation, and machine learning as long-term predictor. More specifically, a dynamic occupancy grid map is utilized as input to a deep convolutional neural network. This yields the advantage of using spatially distributed velocity estimates from a single time step for prediction, rather than a raw data sequence, alleviating common problems dealing with input time series of multiple sensors. Furthermore, convolutional neural networks have the inherent characteristic of using context information, enabling the implicit modeling of road user interaction. Pixel-wise balancing is applied in the loss function counteracting the extreme imbalance between static and dynamic cells. One of the major advantages is the unsupervised learning character due to fully automatic label generation. The presented algorithm is trained and evaluated on multiple hours of recorded sensor data and compared to Monte-Carlo simulation.

A survey of deep learning techniques for autonomous driving

“…Deep learning is used in the context of occupancy maps either for dynamic objects detection and tracking (Ondruska, Dequaire, Wang, & Posner, ), probabilistic estimation of the occupancy map surrounding the vehicle (Hoermann, Bach, & Dietmayer, ; Ramos, Gehrig, Pinggera, Franke, & Rother, ), or for deriving the driving scene context (Marina et al, ; Seeger, Müller, & Schwarz, ). In the latter case, the OG is constructed by accumulating data over time, whereas a deep neural net is used to label the environment into driving context classes, such as highway driving, parking area, or inner‐city driving.…”

“…Deep learning is used in the context of occupancy maps either for dynamic objects detection and tracking (Ondruska, Dequaire, Wang, & Posner, 2016), probabilistic estimation of the occupancy map surrounding the vehicle (Hoermann, Bach, & Dietmayer, 2017;Ramos, Gehrig, Pinggera, Franke, & Rother, 2016), or for deriving the driving scene context Seeger, Müller, & Schwarz, 2016). In the latter case, the OG is constructed by accumulating data over time, whereas a deep neural net is used to F I G U R E 5 Semantic segmentation performance comparison on the CityScapes data set (Cityscapes, 2018 Occupancy maps represent an in-vehicle virtual environment, integrating perceptual information in a form better suited for path planning and motion control.…”

Long-Term Occupancy Grid Prediction Using Recurrent Neural Networks

“…To obtain the velocity ground truth, we generate two binary masks for each time step, one for the static and one for the dynamic environment. The classification of static cells is based on the algorithm described in [7], which relies on the occupancy probabilities of the DOGM. Here, the main idea is to observe the occupancy probability of a single cell for a longer period of time and classify this cell, based on the variation of the occupancy value.…”

“…Although the complete model is trained with supervision, we propose a pipeline for automatic label generation. To that end, we collect large amount of laser measurements that are pre-processed and then labeled with the support of existing algorithms [5], [7], [8].…”

“…The task of predicting future grid maps, as presented in [7], is closely related to the estimation of dynamics. The grid prediction approaches in [7] and [18] rely on dynamic occupancy grid maps input, which simplifies the prediction task by providing velocity estimates of each cell. In contrast, we aim to estimate dynamic occupancy grid maps using only measurement grid maps as input in this work.…”