Photon-Efficient 3D Imaging with A Non-local Neural Network

Peng, Jiayong; Xiong, Zhiwei; Huang, Xin; Li, Zhengping; Liu, Dong; Xu, Feihu

doi:10.1007/978-3-030-58539-6_14

Cited by 29 publications

(60 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where G(x ; •, •) is the gamma distribution with shape and scale parameters, Q is a function of y n and σ2 := σ 2 /s n . To handle high noise in Lidar data, it is common to incorporate multiscale information, as is done in statistical methods [7], [21] as well as deep learning works [17], [18], [20]. We employ a similar multiscale approach, using the fact that lowpass filtered histograms (resulting in summing neighbouring pixels) still follow a Poisson distribution.…”

Section: Multiscale Observation Modelmentioning

confidence: 99%

“…The proposed network also includes soft attention to improve the 3D object reconstruction by considering local spatial correlations. Results on simulated and real data show the benefit of this model when compared to the stateof-the-art learning-based algorithms [17], [18], as it preserves surface edges, has a lower computational cost (in terms of memory or computational time) and provides uncertainty maps on the predicted depth. The uncertainty maps are obtained by connection to the underlying Bayesian method [21] without additional complexity, while some previous works [30], [31] require multiple passes of inference and averaging steps to predict the uncertainty of the network's outputs.…”

Section: Introductionmentioning

confidence: 99%

“…Lindell et al [17] first proposed an end-toend deep learning model which infers depth profiles from Lidar data. Peng et al [18] suggested a non-local network and showed a clear benefit for low photon and high noise cases. Despite their excellent performance on challenging data these deep learning methods lack interpretability, require long running times for high dimensional Lidar data and can present over-smoothing artifacts around 3D surface boundaries.…”

Section: Introductionmentioning

confidence: 99%

“…Using multiscale information to process single-photon data is an essential component of several state-of-the-art 3D Lidar reconstruction algorithms. This was exploited in the proposed network which requires an initial estimate of few multiscale depths as input, instead of the large histogram data cube as in [17], [18]. In this way, the knowledge of the system impulse response function (IRF) is exploited to generate the multiscale depths and the high dimensional data is compressed to the essential information.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

A Bayesian Based Deep Unrolling Algorithm for Single-Photon Lidar Systems

Koo,

Halimi,

McLaughlin

2022

Preprint

View full text Add to dashboard Cite

Deploying 3D single-photon Lidar imaging in real world applications faces multiple challenges including imaging in high noise environments. Several algorithms have been proposed to address these issues based on statistical or learningbased frameworks. Statistical methods provide rich information about the inferred parameters but are limited by the assumed model correlation structures, while deep learning methods show state-of-the-art performance but limited inference guarantees, preventing their extended use in critical applications. This paper unrolls a statistical Bayesian algorithm into a new deep learning architecture for robust image reconstruction from single-photon Lidar data, i.e., the algorithm's iterative steps are converted into neural network layers. The resulting algorithm benefits from the advantages of both statistical and learning based frameworks, providing best estimates with improved network interpretability. Compared to existing learning-based solutions, the proposed architecture requires a reduced number of trainable parameters, is more robust to noise and mismodelling effects, and provides richer information about the estimates including uncertainty measures. Results on synthetic and real data show competitive results regarding the quality of the inference and computational complexity when compared to state-of-the-art algorithms.

show abstract

Section: Multiscale Observation Modelmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Bayesian Based Deep Unrolling Algorithm for Single-Photon Lidar Systems

Koo,

Halimi,

McLaughlin

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…This capability plays important roles for target recognition and identification over long ranges. Meanwhile, advanced computational algorithms have permitted 3D imaging with a small number of photons, i.e., one photon per pixel [27][28][29][30][31][32][33][34]. The combination of the singlephoton LiDAR and photon-efficient imaging algorithms has witnessed the realization of long-range imaging at tens of kilometers [16,17].…”

Section: Introductionmentioning

confidence: 99%

All-Time Single-Photon 3D Imaging Over 21 km

Li¹,

Huang²,

Cao³

et al. 2019

Conference on Lasers and Electro-Optics

Self Cite

View full text Add to dashboard Cite

Long-range active imaging has widespread applications in remote sensing and target recognition. Singlephoton light detection and ranging (LiDAR) has been shown to have high sensitivity and temporal resolution. On the application front, however, the operating range of practical single-photon LiDAR systems is limited to about tens of kilometers over the Earth's atmosphere, mainly due to the weak echo signal mixed with high background noise. Here, we present a compact coaxial single-photon LiDAR system that is capable of realizing 3D imaging at up to 201.5 km. This is achieved by using high-efficiency optical devices for collection and detection, and a new noise-suppression technique that is efficient for longrange applications. We show that photon-efficient computational algorithms enable accurate 3D imaging over hundreds of kilometers with as few as 0.44 signal photon per pixel (PPP). The results represent a significant step towards practical low-power LiDAR over extra-long ranges.

show abstract

Neuromorphic Computing for Compact LiDAR Systems

Delic

Afshar

2023

More-Than-Moore Devices and Integration for Semiconductors

View full text Add to dashboard Cite

Photon-Efficient 3D Imaging with A Non-local Neural Network

Cited by 29 publications

References 44 publications

A Bayesian Based Deep Unrolling Algorithm for Single-Photon Lidar Systems

A Bayesian Based Deep Unrolling Algorithm for Single-Photon Lidar Systems

All-Time Single-Photon 3D Imaging Over 21 km

Neuromorphic Computing for Compact LiDAR Systems

Contact Info

Product

Resources

About