Photon-Efficient Non-Line-of-Sight Imaging

Liu, Jianjiang; Zhou, Yijun; Huang, Xin; Li, Zhengping; Xu, Feihu

doi:10.1109/tci.2022.3194721

Cited by 11 publications

(5 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[6][7][8] NLOS imaging technology can utilize arbitrary walls as mirrors and holds the potential to revolutionize many critical applications, such as medical imaging, autonomous driving, remote sensing, and search and rescue operations. [9][10][11][12][13][14] The NLOS reconstruction problem is an inverse mathematical problem, aimed at recovering the hidden scene from the detected signal. Several challenges exist in NLOS imaging reconstruction.…”

Section: Introductionmentioning

confidence: 99%

“…Detectors, such as a single-photon avalanche diode (SPAD) or a conventional camera, receive speckle images, which can recover shape, location, and albedo of the hidden object from multiple diffuse lights 6 – 8 NLOS imaging technology can utilize arbitrary walls as mirrors and holds the potential to revolutionize many critical applications, such as medical imaging, autonomous driving, remote sensing, and search and rescue operations 9 – 14 …”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Physics-constrained deep-inverse point spread function model: toward non-line-of-sight imaging reconstruction

Wu,

Huang,

Lin

et al. 2024

Adv. Photon. Nexus

View full text Add to dashboard Cite

Non-line-of-sight (NLOS) imaging has emerged as a prominent technique for reconstructing obscured objects from images that undergo multiple diffuse reflections. This imaging method has garnered significant attention in diverse domains, including remote sensing, rescue operations, and intelligent driving, due to its wide-ranging potential applications. Nevertheless, accurately modeling the incident light direction, which carries energy and is captured by the detector amidst random diffuse reflection directions, poses a considerable challenge. This challenge hinders the acquisition of precise forward and inverse physical models for NLOS imaging, which are crucial for achieving high-quality reconstructions. In this study, we propose a point spread function (PSF) model for the NLOS imaging system utilizing ray tracing with random angles. Furthermore, we introduce a reconstruction method, termed the physics-constrained inverse network (PCIN), which establishes an accurate PSF model and inverse physical model by leveraging the interplay between PSF constraints and the optimization of a convolutional neural network. The PCIN approach initializes the parameters randomly, guided by the constraints of the forward PSF model, thereby obviating the need for extensive training data sets, as required by traditional deep-learning methods. Through alternating iteration and gradient descent algorithms, we iteratively optimize the diffuse reflection angles in the PSF model and the neural network parameters. The results demonstrate that PCIN achieves efficient data utilization by not necessitating a large number of actual ground data groups. Moreover, the experimental findings confirm that the proposed method effectively restores the hidden object features with high accuracy.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Physics-constrained deep-inverse point spread function model: toward non-line-of-sight imaging reconstruction

Wu,

Huang,

Lin

et al. 2024

Adv. Photon. Nexus

View full text Add to dashboard Cite

show abstract

“…As a result, SPAD devices have found applications in various fields requiring high-resolution weak light timing signals. These applications include direct time-of-flight single-photon imaging and high-performance LiDAR [4][5][6][7], non-line-of-sight imaging [8][9][10],…”

Section: Introductionmentioning

confidence: 99%

Designing a Sub-20V Breakdown Voltage SPAD With Standard CMOS Technology and n/p-well Structure

Wu,

Liu

2024

IEEE Photonics J.

View full text Add to dashboard Cite

We have proposed a structural design for a single photon avalanche diode with a low breakdown voltage. This diode is fabricated using Taiwan Semiconductor Manufacturing Company (TSMC) 0.18 µm HV CMOS technology, and it can maintain a high operating excess voltage in an n-on-p design without requiring any additional customized well layers. The n-on-p type device is particularly advantageous for a 3D-stacked backside illuminated structure and offers excellent photon detection capabilities at longer wavelengths. By incorporating a high doping concentration PDD well layer, we can significantly increase the excess bias, resulting in enhanced photon detection probability in the near-infrared wavelength range, all while maintaining a lower voltage due to a reduction in breakdown voltage. This design also leads to power consumption savings. As a result, our designed device is well-suited for consumer applications such as 3D image rendering and Lidar technology.

show abstract

“…To break this restriction, non‐line‐of‐sight (NLOS) imaging analyzes the diffuse reflection from a relay wall to image hidden objects [FVW20, MSS*19], which has broad applications in many fields, such as medical imaging, autonomous driving, and robotic vision [LWK19, SMBG19, SOG18, XST*18]. With the rapid development of photon‐sensitive sensors and imaging algorithms, most current NLOS imaging techniques utilize inverse physical models [LZH*22, LWL*21] that are constructed with active or passive lighting and reconstruction algorithms to recover hidden scenes [BZT*15, GWV*12, LGLM*19, VWG*12, WZH*21, WLH*21]. Most recently, deep learning algorithms for NLOS imaging have received a lot of attention [GCHWI20, CWK*20].…”

Section: Introductionmentioning

confidence: 99%

Multi‐scale Iterative Model‐guided Unfolding Network for NLOS Reconstruction

Su,

Hong,

et al. 2023

Computer Graphics Forum

Self Cite

View full text Add to dashboard Cite

Non‐line‐of‐sight (NLOS) imaging can reconstruct hidden objects by analyzing diffuse reflection of relay surfaces, and is potentially used in autonomous driving, medical imaging and national defense. Despite the challenges of low signal‐to‐noise ratio (SNR) and ill‐conditioned problem, NLOS imaging has developed rapidly in recent years. While deep neural networks have achieved impressive success in NLOS imaging, most of them lack flexibility when dealing with multiple spatial‐temporal resolution and multi‐scene images in practical applications. To bridge the gap between learning methods and physical priors, we present a novel end‐to‐end Multi‐scale Iterative Model‐guided Unfolding (MIMU), with superior performance and strong flexibility. Furthermore, we overcome the lack of real training data with a general architecture that can be trained in simulation. Unlike existing encoder‐decoder architectures and generative adversarial networks, the proposed method allows for only one trained model adaptive for various dimensions, such as various sampling time resolution, various spatial resolution and multiple channels for colorful scenes. Simulation and real‐data experiments verify that the proposed method achieves better reconstruction results both in quality and quantity than existing methods.

show abstract

Photon-Efficient Non-Line-of-Sight Imaging

Cited by 11 publications

References 49 publications

Physics-constrained deep-inverse point spread function model: toward non-line-of-sight imaging reconstruction

Physics-constrained deep-inverse point spread function model: toward non-line-of-sight imaging reconstruction

Designing a Sub-20V Breakdown Voltage SPAD With Standard CMOS Technology and n/p-well Structure

Multi‐scale Iterative Model‐guided Unfolding Network for NLOS Reconstruction

Contact Info

Product

Resources

About