Epipolar time-of-flight imaging

Achar, Supreeth; Bartels, Joseph; Whittaker, William; Kutulakos, Kiriakos N.; Narasimhan, Srinivasa G.

doi:10.1145/3072959.3073686

Cited by 81 publications

(40 citation statements)

References 15 publications

(2 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The theoretical criteria derived here can be used in conjunction with these hardware architectures for optimal LiDAR design. Active 3D imaging in sunlight: Prior work in the structured light and time-of-flight literature proposes various coding and illumination schemes to address the problem of low signal-to-noise ratios (SNR) due to strong ambient light [19,12,23,1]. The present work deals with a different problem of optimal photon detection for SPAD-based pulsed time-of-flight.…”

Section: Related Workmentioning

confidence: 99%

Photon-Flooded Single-Photon 3D Cameras

Gupta

Ingle

Velten

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

Single-photon avalanche diodes (SPADs) are starting to play a pivotal role in the development of photon-efficient, long-range LiDAR systems.However, due to nonlinearities in their image formation model, a high photon flux (e.g., due to strong sunlight) leads to distortion of the incident temporal waveform, and potentially, large depth errors. Operating SPADs in low flux regimes can mitigate these distortions, but, often requires attenuating the signal and thus, results in low signal-to-noise ratio. In this paper, we address the following basic question: what is the optimal photon flux that a SPAD-based LiDAR should be operated in? We derive a closed form expression for the optimal flux, which is quasi-depth-invariant, and depends on the ambient light strength. The optimal flux is lower than what a SPAD typically measures in real world scenarios, but surprisingly, considerably higher than what is conventionally suggested for avoiding distortions. We propose a simple, adaptive approach for achieving the optimal flux by attenuating incident flux based on an estimate of ambient light strength. Using extensive simulations and a hardware prototype, we show that the optimal flux criterion holds for several depth estimators, under a wide range of illumination conditions. †

show abstract

Section: Related Workmentioning

confidence: 99%

Photon-Flooded Single-Photon 3D Cameras

Gupta

Ingle

Velten

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

show abstract

“…Synchronization with camera exposure has allowed for reconstruction in the face of strong ambient light [1]. Transient imaging is possible using ultra-fast lasers [42], and has recently been demonstrated using mobile off-the-shelf devices [14].…”

Section: Related Workmentioning

confidence: 99%

“…Recent work has addressed some aspects of TOF energy efficiency with novel illumination encodings. For example, by synchronizing illumination patterns to match sensor exposures [1], low-power reconstruction can occur for scenes with significant ambient light. Additionally, spatio-temporal encodings have been shown to be efficient for both structured light illumination [30] and TOF illumination as well [29].…”

mentioning

confidence: 99%

Directionally Controlled Time-of-Flight Ranging for Mobile Sensing Platforms

Tasneem

Wang

Xie

et al. 2018

Robotics: Science and Systems XIV

View full text Add to dashboard Cite

Abstract-Scanning time-of-flight (TOF) sensors obtain depth measurements by directing modulated light beams across a scene. We demonstrate that control of the directional scanning patterns can enable novel algorithms and applications. Our analysis occurs entirely in the angular domain and consists of two ideas. First, we show how to exploit the angular support of the light beam to improve reconstruction results. Second, we describe how to control the light beam direction in a way that maximizes a well-known information theoretic measure. Using these two ideas, we demonstrate novel applications such as adaptive TOF sensing, LIDAR zoom, LIDAR edge sensing for gradient-based reconstruction and energy efficient LIDAR scanning. Our contributions can apply equally to sensors using mechanical, optoelectronic or MEMS-based approaches to modulate the light beam, and we show results here on a MEMS mirror-based LIDAR system. In short, we describe new adaptive directionally controlled TOF sensing algorithms which can impact mobile sensing platforms such as robots, wearable devices and IoT nodes. Creating TOF sensors for personal drones, VR/AR glasses, IoT nodes and other miniature platforms would require transcending the energy constraints due to limited battery capacity. Recent work has addressed some aspects of TOF energy efficiency with novel illumination encodings. For example, by synchronizing illumination patterns to match sensor exposures [1], low-power reconstruction can occur for scenes with significant ambient light. Additionally, spatio-temporal encodings have been shown to be efficient for both structured light illumination [30] and TOF illumination as well [29]. I. INTRODUCTIONIn this paper, we demonstrate new efficiencies that are possible with angular control of a TOF sensor. We demonstrate this with a single LIDAR beam reflected off a microelectromechanical (MEMS) mirror. The voltages that control the MEMS actuators allow analog (continuous) TOF sensing angles. As a modulator, MEMS mirrors have well-known advantages of high-speed and fast response to control [32].

show abstract

“…Active depth cameras, such as scanning lidar systems, have not only become a cornerstone imaging modality for autonomous driving and robotics, but are emerging in applications across disciplines, including autonomous drones, remote sensing, human-computer interaction, and augmented or virtual reality. Depth cameras that provide dense range allow for dense scene reconstructions [26] when combined with color cameras, including correlation time-offlight cameras (C-ToF) [19,30,33] such as Microsoft's Kinect One, or structured light cameras [1,42,43,49]. These acquisition systems facilitate the collection of largescale RGB-D data sets that fuel research on core computer vision problems, including scene understanding [23,53] and action recognition [40].…”

Section: Introductionmentioning

confidence: 99%

Gated2Depth: Real-Time Dense Lidar From Gated Images

Gruber

Julca-Aguilar²,

Bijelic

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

We present an imaging framework which converts three images from a gated camera into high-resolution depth maps with depth accuracy comparable to pulsed lidar measurements. Existing scanning lidar systems achieve low spatial resolution at large ranges due to mechanicallylimited angular sampling rates, restricting scene understanding tasks to close-range clusters with dense sampling. Moreover, today's pulsed lidar scanners suffer from high cost, power consumption, large form-factors, and they fail in the presence of strong backscatter. We depart from point scanning and demonstrate that it is possible to turn a low-cost CMOS gated imager into a dense depth camera with at least 80 m range -by learning depth from three gated images. The proposed architecture exploits semantic context across gated slices, and is trained on a synthetic discriminator loss without the need of dense depth labels. The proposed replacement for scanning lidar systems is real-time, handles back-scatter and provides dense depth at long ranges. We validate our approach in simulation and on real-world data acquired over 4,000 km driving in northern Europe. Data and code are available at

show abstract

Epipolar time-of-flight imaging

Cited by 81 publications

References 15 publications

Photon-Flooded Single-Photon 3D Cameras

Photon-Flooded Single-Photon 3D Cameras

Directionally Controlled Time-of-Flight Ranging for Mobile Sensing Platforms

Gated2Depth: Real-Time Dense Lidar From Gated Images

Contact Info

Product

Resources

About