Super-Pixel Sampler: a Data-driven Approach for Depth Sampling and Reconstruction

Wolff, Adam; Praisler, Shachar; Ilya, Tcenov,; Gilboa, Guy

doi:10.1109/icra40945.2020.9197191

Cited by 9 publications

(15 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Signal Expansion. The expansion idea has shown in tasks like superpixel segmentation [1,42,46,36], depth completion, and depth sampling [15,22,45]. Superpixel aggregates pixels with similar semantics, but they do not imply similar depth values.…”

Section: Related Workmentioning

confidence: 99%

$S^3$: Learnable Sparse Signal Superdensity for Guided Depth Estimation

Huang

Liu

et al. 2021

Preprint

View full text Add to dashboard Cite

Dense depth estimation plays a key role in multiple applications such as robotics, 3D reconstruction, and augmented reality. While sparse signal, e.g., LiDAR and Radar, has been leveraged as guidance for enhancing dense depth estimation, the improvement is limited due to its low density and imbalanced distribution. To maximize the utility from the sparse source, we propose Sparse Signal Superdensity (S 3 ) technique, which expands the depth value from sparse cues while estimating the confidence of expanded region. The proposed S 3 can be applied to various guided depth estimation approaches and trained end-to-end at different stages, including input, cost volume and output. Extensive experiments demonstrate the effectiveness, robustness, and flexibility of the S 3 technique on LiDAR and Radar signal.

show abstract

Section: Related Workmentioning

confidence: 99%

$S^3$: Learnable Sparse Signal Superdensity for Guided Depth Estimation

Huang

Liu

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…This requires the adaptive sampling network to predict sampling locations ((x, y) coordinates) directly and the sampling process be differentiable. For the RGB and sparse depth adaptive sampling task, Wolff et al [58] use the SLIC superpixel technique [1] to segment the RGB image and sample the depth map at the center of mass of each superpixel. A bilateral filtering based reconstruction algorithm is proposed to reconstruct the depth map.…”

Section: Sampling Mask Optimizationmentioning

confidence: 99%

“…The sampling and reconstruction methods are not optimized jointly, leaving room for improvement. In this paper, we show that jointly training recent DL based superpixel sampling networks [23,58] and depth estimation networks [7,13,47,9,48,36,38,54] could adapt the sampling network to depth estimation and obtain improved reconstruction accuracy. Bergman et al [4] warp an uniform sampling grid to generate the adaptive sampling mask.…”

Section: Sampling Mask Optimizationmentioning

confidence: 99%

“…Existing irregular sampling techniques [13,5] and adaptive depth sampling methods [4,58] explicitly or implicitly make sampling points evenly distributed spatially. Such prior is important when the sampling rate is low.…”

Section: Sampling Mask Generation Network N Etmmentioning

confidence: 99%

“…Such prior is important when the sampling rate is low. Inspired by the SLIC superpixel [1] based adaptive sampling method [58], we propose to utilize recent DL based superpixel networks [60,23] as N etM . As demonstrated in Fig-…”

Section: Sampling Mask Generation Network N Etmmentioning

confidence: 99%

See 2 more Smart Citations

Adaptive Illumination based Depth Sensing using Deep Superpixel and Soft Sampling Approximation

Dai¹,

Li²,

Cossairt³

et al. 2021

Preprint

View full text Add to dashboard Cite

Dense depth map capture is challenging in existing active sparse illumination based depth acquisition techniques, such as LiDAR. Various techniques have been proposed to estimate a dense depth map based on fusion of the sparse depth map measurement with the RGB image. Recent advances in hardware enable adaptive depth measurements resulting in further improvement of the dense depth map estimation. In this paper, we study the topic of estimating dense depth from depth sampling. The adaptive sparse depth sampling network is jointly trained with a fusion network of an RGB image and sparse depth, to generate optimal adaptive sampling masks. We show that such adaptive sampling masks can generalize well to many RGB and sparse depth fusion algorithms under a variety of sampling rates (as low as 0.0625%). The proposed adaptive sampling method is fully differentiable and flexible to be trained endto-end with upstream perception algorithms.

show abstract