Non-Line-of-Sight Reconstruction using Efficient Transient Rendering

Iseringhausen, Julian; Hullin, Matthias B.

doi:10.48550/arxiv.1809.08044

Cited by 2 publications

(6 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several techniques have been developed in recent years for rendering transient light transport [16,17,26,34,35,38]. Due to its accuracy and efficiency, we opt for the technique introduced by Iseringhausen et al [16], which can be adapted to various capture geometries including the confocal setting [29]. As the renderer accumulates light contributions per triangle of a meshed object, we triangulate our depth samples and feed them to the renderer.…”

Section: Transient Renderingmentioning

confidence: 99%

“…Since most publicly available real-world datasets have been acquired by single photon avalanche detectors (SPADs), we seek to extend our for- [29]. Center: A rendered pixel computed by using the forward model [16]. Right: Rendered pixel after applying our SPAD model.…”

Section: Sensor Modelmentioning

confidence: 99%

“…In this setting, which we call the non-line-of-sight (NLoS) reconstruction problem, the loss of temporal and directional information caused by multiple bounces across more or less diffuse surfaces means that the problem has to be approached in a drastically different manner. Among the techniques proposed in literature for reconstructing object geometry and reflectance from three-bounce indirect measurements are combinatorial approaches [19], parametric model fits [16,19,20,28,30,35], backprojection [2,21,36], inverse filtering approaches [13,29] and space-time diffraction integrals [24,25]. For the purpose of this paper, we focus on such active-illumination, time-of-flight input despite the fact that steady-state or even purely passive approaches have also been demonstrated [3,5,18,20].…”

Section: Introductionmentioning

confidence: 99%

“…In this paper, we build upon a highly efficient transient renderer [16] to generate large amounts of training data for solving the NLoS reconstruction problem using deep neural networks. Our contributions to arrive at a full deep learning solution include:…”

Section: Introductionmentioning

confidence: 99%

“…Left: A real pixel measured with a SPAD sensor[29]. Center: A rendered pixel computed by using the forward model[16]. Right: Rendered pixel after applying our SPAD model.…”

mentioning

confidence: 99%

See 4 more Smart Citations

Deep Non-Line-of-Sight Reconstruction

Chopite¹,

Hullin²,

Wand³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

The recent years have seen a surge of interest in methods for imaging beyond the direct line of sight. The most prominent techniques rely on time-resolved optical impulse responses, obtained by illuminating a diffuse wall with an ultrashort light pulse and observing multi-bounce indirect reflections with an ultrafast time-resolved imager. Reconstruction of geometry from such data, however, is a complex non-linear inverse problem that comes with substantial computational demands. In this paper, we employ convolutional feed-forward networks for solving the reconstruction problem efficiently while maintaining good reconstruction quality. Specifically, we devise a tailored autoencoder architecture, trained end-to-end, that maps transient images directly to a depth map representation. Training is done using an efficient transient renderer for diffuse three-bounce indirect light transport that enables the quick generation of large amounts of training data for the network. We examine the performance of our method on a variety of synthetic and experimental datasets and its dependency on the choice of training data and augmentation strategies, as well as architectural features. We demonstrate that our feed-forward network, even though it is trained solely on synthetic data, generalizes to measured data from SPAD sensors and is able to obtain results that are competitive with model-based reconstruction methods.

show abstract

Section: Transient Renderingmentioning

confidence: 99%

Section: Sensor Modelmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

“…Left: A real pixel measured with a SPAD sensor[29]. Center: A rendered pixel computed by using the forward model[16]. Right: Rendered pixel after applying our SPAD model.…”

mentioning

confidence: 99%

See 3 more Smart Citations

Deep Non-Line-of-Sight Reconstruction

Chopite¹,

Hullin²,

Wand³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Non-Line-of-Sight Imaging

Faccio¹

2019

Optics & Photonics News

View full text Add to dashboard Cite

Emerging single-photon-sensitive sensors combined with advanced inverse methods to process picosecond-accurate time-stamped photon counts have given rise to unprecedented imaging capabilities. Rather than imaging photons that travel along direct paths from a source to an object and back to the detector, non-line-of-sight (NLOS) imaging approaches analyze photons scattered from multiple surfaces that travel along indirect light paths to estimate 3D images of scenes outside the direct line of sight of a camera, hidden by a wall or other obstacles. Here we review recent advances in the field of NLOS imaging, discussing how to see around corners and future prospects for the field.

show abstract

Non-Line-of-Sight Reconstruction using Efficient Transient Rendering

Cited by 2 publications

References 0 publications

Deep Non-Line-of-Sight Reconstruction

Deep Non-Line-of-Sight Reconstruction

Non-Line-of-Sight Imaging

Contact Info

Product

Resources

About