Non-line-of-sight Imaging with Partial Occluders and Surface Normals

Abstract

Fig. 1. Non-line-of-sight (NLOS) imaging aims at recovering the shape and albedo of objects hidden from a camera or a light source. Using ultra-fast pulsed illumination and single photon detectors, the light transport in the scene is sampled for visible objects (left). The global illumination components of these time-resolved measurements (A,E) contain sufficient information to estimate the shape of hidden objects (B,C). Using a novel formulation for NLOS light transport that models partial occlusions of hidden objects (D) via visibility terms (F), we demonstrate higher-fidelity reconstructions (C) than previous approaches to NLOS imaging (B).Imaging objects obscured by occluders is a significant challenge for many applications. A camera that could "see around corners" could help improve navigation and mapping capabilities of autonomous vehicles or make search and rescue missions more effective. Time-resolved single-photon imaging systems have recently been demonstrated to record optical information of a scene that can lead to an estimation of the shape and reflectance of objects hidden from the line of sight of a camera. However, existing nonline-of-sight (NLOS) reconstruction algorithms have been constrained in the types of light transport effects they model for the hidden scene parts. We introduce a factored NLOS light transport representation that accounts for partial occlusions and surface normals. Based on this model, we develop a factorization approach for inverse time-resolved light transport and demonstrate high-fidelity NLOS reconstructions for challenging scenes both in simulation and with an experimental NLOS imaging system.

Phasor field diffraction based reconstruction for fast non-line-of-sight imaging systems

“…Full color NLOS imaging with single pixel photomultiplier tube combined with a mask 23,24 has also been demonstrated. Further work includes real-time transient imaging for amplitude modulated continuous wave lidar applications 25 , analysis of missing features based on time-resolved NLOS measurements 26 , convolutional approximations to incorporate priors into FBP 27 , occlusion-aided NLOS imaging using SPADs 28,29 , Bayesian statistics reconstruction to account for random errors 30 , temporal focusing for a hidden volume of interest by altering the time delay profile of the hardware illumination 31 , and a database for NLOS imaging problems with different acquisition schemes 32 . Reconstruction times for all these methods remain in the minutes to hours range even for small scenes of less than a meter in diameter.…”

Occlusion-based computational periscopy with consumer cameras

“…In general, this has been achieved using measurements of properties of the light scattered onto visible surfaces from the hidden scene. For example, these methods have relied on measurements of time-of-flight, [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17] coherence, [18][19][20] or even intensity-only [21][22][23][24][25][26] information. NLOS imaging using acoustic 27 and long-wave infrared 28 waves have also been recent lines of work.…”

Non-line-of-sight imaging using phasor-field virtual wave optics

“…Recent NLOS reconstruction methods are based on heuristic filtered backprojection 2,3,6,7,21 , or attempt to compute inverse operators of simplified forward light transport models 5,9,19 . These simplified models do not take into account multiple scattering, surfaces with anisotropic reflectance or, with a few exceptions 19 , occlusions and clutter. Moreover, the depth range that can be recovered is also limited, partially due to the difference in intensity between first-and higher-order reflections.…”