Adaptive LiDAR Sampling and Depth Completion Using Ensemble Variance

Gofer, Eyal; Praisler, Shachar; Gilboa, Guy

doi:10.1109/tip.2021.3120042

Cited by 14 publications

(9 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, to obtain an estimation of y we need to sample it. One may resort to iterative sampling, as suggested in [17]. Here we propose to use RGB side information, in order to estimate locations of high uncertainty, which require denser sampling.…”

Section: Key Conceptmentioning

confidence: 99%

“…Limitations of the proposed framework: We note there are a few assumptions, related to the LiDAR data, which simplify the problem and was assumed by us and by recent papers in the field [2,17,47]. First, it is assumed that the adaptive LiDAR can sample depth at any specified location (for KITTI dataset the ground truth is not full and the assumption is weaker).…”

Section: Estimating Q From Rgbmentioning

confidence: 99%

“…They target the extreme low-budget regime (and show that at sparsity above ≈ 0.1% their advantage over fixed sampling diminishes). Gofer et al [17] use the variance (disagreement) between an ensemble of predictors (neural networks or random forests) to extract the most relevant sampling locations. They focus on minimizing RMSE (root mean square error), which is closely related to variance.…”

Section: Introductionmentioning

confidence: 99%

“…They focus on minimizing RMSE (root mean square error), which is closely related to variance. A strong feature in [17] is their ability to work also without guidance. Our work, however, aims to go beyond RMSE, which may not be the most suitable measure to assess depth reconstruction.…”

Section: Introductionmentioning

confidence: 99%

“…Paper Goal: In our work we suggest a generic guidance measure for efficient adaptive depth sampling, which allows to optimize a wide variety of quality measures of a given depth completion method. Based on this guidance, a simple algorithm is proposed which compares favourably to [47] and [17]. The measure can be further used by more sophisticated algorithms or as an additional loss-term in network architectures.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

How to Guide Adaptive Depth Sampling?

Ilya¹,

Gilboa²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Recent advances in depth sensing technologies allow fast electronic maneuvering of the laser beam, as opposed to fixed mechanical rotations. This will enable future sensors, in principle, to vary in real-time the sampling pattern. We examine here the abstract problem of whether adapting the sampling pattern for a given frame can reduce the reconstruction error or allow a sparser pattern. We propose a constructive generic method to guide adaptive depth sampling algorithms.Given a sampling budget B, a depth predictor P and a desired quality measure M , we propose an Importance Map that highlights important sampling locations. This map is defined for a given frame as the per-pixel expected value of M produced by the predictor P , given a pattern of B random samples. This map can be well estimated in a training phase. We show that a neural network can learn to produce a highly faithful Importance Map, given an RGB image. We then suggest an algorithm to produce a sampling pattern for the scene, which is denser in regions that are harder to reconstruct. The sampling strategy of our modular framework can be adjusted according to hardware limitations, type of depth predictor, and any custom reconstruction error measure that should be minimized. We validate through simulations that our approach outperforms grid and random sampling patterns as well as recent state-of-the-art adaptive algorithms.

show abstract

Section: Key Conceptmentioning

confidence: 99%

Section: Estimating Q From Rgbmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations