Sparse sensing for resource-constrained depth reconstruction

Abstract

We address the following question: is it possible to reconstruct the geometry of an unknown environment using sparse and incomplete depth measurements? This problem is relevant for a resource-constrained robot that has to navigate and map an environment, but does not have enough on-board power or payload to carry a traditional depth sensor (e.g., a 3D lidar) and can only acquire few (point-wise) depth measurements. In general, reconstruction from incomplete data is not possible, but when the robot operates in man-made environments, the depth exhibits some regularity (e.g., many planar surfaces with few edges); we leverage this regularity to infer depth from incomplete measurements. Our formulation bridges robotic perception with the compressive sensing literature in signal processing. We exploit this connection to provide formal results on exact depth recovery in 2D and 3D problems. Taking advantage of our specific sensing modality, we also prove novel and more powerful results to completely characterize the geometry of the signals that we can reconstruct. Our results directly translate to practical algorithms for depth reconstruction; these algorithms are simple (they reduce to solving a linear program), and robust to noise. We test our algorithms on real and simulated data, and show that they enable accurate depth reconstruction from a handful of measurements, and perform well even when the assumption of structured environment is violated.

Event Guided Depth Sensing

“…While we conservatively sample the scene, a natural question to ask is "can we recover the complete depth profile from sparse depth measurements?" Several works address the problem of full scene depth estimation from sparse samples [20][21][22][23]. Recently, [24] formulated the depth com-pletion problem with compressive sensing literature and provided exactness and stability analysis for 3D depth completion.…”

Deep Convolutional Compressed Sensing for LiDAR Depth Completion

“…L.-K. Liu et al [8] built on that by combining wavelets with contourlets and investigated the effect of different sampling patterns. Both methods were out performed by Ma & Karaman [9] who exploit the simple structures of man-made indoor scenes to achieve full depth reconstruction. In contrast to all of these works, our approach learns multi-level convolutional dictionaries from a large dataset of incomplete ground truth depth maps.…”

ROI-Based LiDAR Sampling Algorithm in on-Road Environment for Autonomous Driving

“…5(b). It is noteworthy that the sampling method in [25] is based on a wavelet, and the contour-based reconstruction method performs ineffectively when the sample budget is small (i.e., 1% or 2%) [14]. That is, it is challenging to obtain a reliable gradient image of a scene in an outdoor scenario because generally, its RGB image is complicated and its raw depth image excessively sparse to estimate a reliable gradient map (i.e., 1% ∼ 2% sparse compared to an RGB image).…”