Project to Adapt: Domain Adaptation for Depth Completion from Noisy and Sparse Sensor Data

Abstract

Depth completion aims to predict a dense depth map from a sparse depth input. The acquisition of dense ground truth annotations for depth completion settings can be difficult and, at the same time, a significant domain gap between real LiDAR measurements and synthetic data has prevented from successful training of models in virtual settings. We propose a domain adaptation approach for sparse-to-dense depth completion that is trained from synthetic data, without annotations in the real domain or additional sensors. Our approach simulates the real sensor noise in an RGB + LiDAR set-up, and consists of three modules: simulating the real LiDAR input in the synthetic domain via projections, filtering the real noisy LiDAR for supervision and adapting the synthetic RGB image using a CycleGAN [1] approach. We extensively evaluate these modules against the state-of-the-art in the KITTI depth completion benchmark, showing significant improvements.

CroMo: Cross-Modal Learning for Monocular Depth Estimation

“…Depth completion has been carried out via combining multiple input modalities, for example, a sparse but accurate LiDAR signal in combination with RGB [58]. It is difficult to address sparse signals with CNNs [41] and LiDAR sensors can produce problematic artifacts resulting in unreliable Ground Truth depth estimates [39]. One strategy towards removing dependence on this form of supervision are self-supervision cues however these fall behind supervised pipelines in terms of accuracy [40].…”