Self-Supervised Flow Estimation using Geometric Regularization with Applications to Camera Image and Grid Map Sequences

Abstract

We present a self-supervised approach to estimate flow in camera image and top-view grid map sequences using fully convolutional neural networks in the domain of automated driving. We extend existing approaches for self-supervised optical flow estimation by adding a regularizer expressing motion consistency assuming a static environment. However, as this assumption is violated for other moving traffic participants we also estimate a mask to scale this regularization. Adding a regularization towards motion consistency improves convergence and flow estimation accuracy. Furthermore, we scale the errors due to spatial flow inconsistency by a mask that we derive from the motion mask. This improves accuracy in regions where the flow drastically changes due to a better separation between static and dynamic environment. We apply our approach to optical flow estimation from camera image sequences, validate on odometry estimation and suggest a method to iteratively increase optical flow estimation accuracy using the generated motion masks. Finally, we provide quantitative and qualitative results based on the KITTI odometry and tracking benchmark for scene flow estimation based on grid map sequences. We show that we can improve accuracy and convergence when applying motion and spatial consistency regularization.

Dynamic Occupancy Grid Mapping with Recurrent Neural Networks

“…The work in [5] and [6] adopted this approach and independently propose to only represent the dynamic cells with particles, whereas Nuss et al [7] suggest to define the dynamic grid mapping as a random finite set problem. Recently, several deep learning approaches have utilized grid maps as input data for convolutional neural networks to detect objects [11], [12], separate the occupied area in static or dynamic cells [13], predict future occupancy [14] or use self-supervised scene flow prediction to estimate odometry [15]. Dequaire et al [16] propose an end-to-end trainable recurrent neural network to estimate unoccluded occupancy grid maps, based on raw lidar data as input, but without having velocity estimates.…”

Long-Term Occupancy Grid Prediction Using Recurrent Neural Networks

“…Wirges et al [15] propose a multi-layer grid to encode several features of lidar point clouds for object detection. The publications [16] and [17] propose a self-supervised flow estimation in grid maps to solve the task of future prediction, respectively odometry estimation. The task of predicting future grid maps, as presented in [7], is closely related to the estimation of dynamics.…”