Advances in Single-Photon Lidar for Autonomous Vehicles: Working Principles, Challenges, and Recent Advances

Rapp, Joshua; Tachella, Julián; Altmann, Yoann; McLaughlin, Stephen; Goyal, Vivek K

doi:10.1109/msp.2020.2983772

Cited by 96 publications

(63 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the case of single-photon counting lidar, the dimension d = 1. Typically, we observe a finite dataset X = {x i } n i=1 of n samples which we assume is sampled from the distribution given in (2). Maximum likelihood estimation (MLE) is a traditional parameter estimation method whereby a likelihood function associated with the finite data is maximized with respect to the model parameters, e.g.…”

Section: Summary Statisticsmentioning

confidence: 99%

“…We coin this set the orthogonal frequencies as it defines regions over the interval of the observation model's characteristic function where the signal's contribution is orthogonal to the background's contribution. 1) Sampling Schemes: In order to construct a sketch, we are ultimately interested in retaining sufficient salient information of the characteristic function Ψ π such that we can identify and estimate the unique location and intensity parameters θ of the observation model π(x | θ) defined in (2). It was discussed in Section II that the CF of a probability distribution decays in frequency, i.e.…”

Section: B Sampling the Ecfmentioning

confidence: 99%

“…S INGLE photon light detection and ranging (lidar) has emerged as an important depth imaging technique prevalent in the automobile [1], [2], defence [3] and forestry industries [4]. This modality has the unique advantage of offering very high depth resolution [5] even at long-range scenes using low-power (eye-safe) lasers [6].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Sketching Framework for Reduced Data Transfer in Photon Counting Lidar

Sheehan¹,

Tachella²,

Davies³

2021

IEEE Trans. Comput. Imaging

Self Cite

View full text Add to dashboard Cite

Single-photon lidar has become a prominent tool for depth imaging in recent years. At the core of the technique, the depth of a target is measured by constructing a histogram of time delays between emitted light pulses and detected photon arrivals. A major data processing bottleneck arises on the device when either the number of photons per pixel is large or the resolution of the time-stamp is fine, as both the space requirement and the complexity of the image reconstruction algorithms scale with these parameters. We solve this limiting bottleneck of existing lidar techniques by sampling the characteristic function of the time of flight (ToF) model to build a compressive statistic, a socalled sketch of the time delay distribution, which is sufficient to infer the spatial distance and intensity of the object. The size of the sketch scales with the degrees of freedom of the ToF model (number of objects) and not, fundamentally, with the number of photons or the time-stamp resolution. Moreover, the sketch is highly amenable for on-chip online processing. We show theoretically that the loss of information for compression is controlled and the mean squared error of the inference quickly converges towards the optimal Cramér-Rao bound (i.e. no loss of information) for modest sketch sizes. The proposed compressed single-photon lidar framework is tested and evaluated on real life datasets of complex scenes where it is shown that a compression rate of up-to 150 is achievable in practice without sacrificing the overall resolution of the reconstructed image.

show abstract

Section: Summary Statisticsmentioning

confidence: 99%

Section: B Sampling the Ecfmentioning

confidence: 99%

See 1 more Smart Citation

A Sketching Framework for Reduced Data Transfer in Photon Counting Lidar

Sheehan¹,

Tachella²,

Davies³

2021

IEEE Trans. Comput. Imaging

Self Cite

View full text Add to dashboard Cite

show abstract

“…Three-dimensional (3D) imaging has generated significant interest from the scientific community due to its increasing use in applications such as self-driving autonomous vehicles [1], [2]. Single-photon light detection and ranging (Lidar) is a technology for high resolution 3D imaging, where its high sensitivity and excellent surface-to-surface resolution can provide rich information on the depth profile and reflectivity of observed targets in challenging imaging scenarios.…”

Section: Introductionmentioning

confidence: 99%

Robust and Guided Bayesian Reconstruction of Single-Photon 3D Lidar Data: Application to Multispectral and Underwater Imaging

Halimi¹,

Maccarone²,

Lamb³

et al. 2021

IEEE Trans. Comput. Imaging

Self Cite

View full text Add to dashboard Cite

3D Lidar imaging can be a challenging modality when using multiple wavelengths, or when imaging in high noise environments (e.g., imaging through obscurants). This paper presents a hierarchical Bayesian algorithm for the robust reconstruction of multispectral single-photon Lidar data in such environments. The algorithm exploits multi-scale information to provide robust depth and reflectivity estimates together with their uncertainties to help with decision making. The proposed weightbased strategy allows the use of available guide information that can be obtained by using state-of-the-art learning based algorithms. The proposed Bayesian model and its estimation algorithm are validated on both synthetic and real images showing competitive results regarding the quality of the inferences and the computational complexity when compared to the state-of-the-art algorithms.

show abstract

“…The ability to see around corners would be profoundly useful in numerous fields, from helping see past partial blockages in medical settings to enabling surveillance while remaining undetected. Among the applications of non-line-of-sight (NLOS) imaging with the most potential is autonomous navigation, which could leverage existing sensing hardware to gather information about hidden pedestrians, vehicles, or other potential obstacles and plan safer trajectories through intersections or into occluded spaces 1 . Practical implementation of anticipatory imaging would require fast acquisition and reconstruction of a large-scale scene with a wide field of view (FOV) in order to detect the most significant obstacles with enough time to react safely.…”

Section: Introductionmentioning

confidence: 99%

Seeing around corners with edge-resolved transient imaging

et al. 2020

Self Cite

View full text Add to dashboard Cite

Non-line-of-sight (NLOS) imaging is a rapidly growing field seeking to form images of objects outside the field of view, with potential applications in autonomous navigation, reconnaissance, and even medical imaging. The critical challenge of NLOS imaging is that diffuse reflections scatter light in all directions, resulting in weak signals and a loss of directional information. To address this problem, we propose a method for seeing around corners that derives angular resolution from vertical edges and longitudinal resolution from the temporal response to a pulsed light source. We introduce an acquisition strategy, scene response model, and reconstruction algorithm that enable the formation of 2.5-dimensional representations—a plan view plus heights—and a 180∘ field of view for large-scale scenes. Our experiments demonstrate accurate reconstructions of hidden rooms up to 3 meters in each dimension despite a small scan aperture (1.5-centimeter radius) and only 45 measurement locations.

show abstract

Advances in Single-Photon Lidar for Autonomous Vehicles: Working Principles, Challenges, and Recent Advances

Cited by 96 publications

References 31 publications

A Sketching Framework for Reduced Data Transfer in Photon Counting Lidar

A Sketching Framework for Reduced Data Transfer in Photon Counting Lidar

Robust and Guided Bayesian Reconstruction of Single-Photon 3D Lidar Data: Application to Multispectral and Underwater Imaging

Seeing around corners with edge-resolved transient imaging

Contact Info

Product

Resources

About