Homogeneous codes for energy-efficient illumination and imaging

Abstract

Figure 1: Imagine trying to acquire live video (30fps) of structured-light patterns as they are being projected onto a compact fluorescent bulb that has already been turned on (rated 1600 Lumens), or onto a face in bright sunlight (80 kLux)-with a 5-Lumen projector. One of our two prototypes, shown in (a), achieves this with an off-the-shelf laser projector and a CMOS camera with an ordinary lens. We then used it to capture the video frames in (b) and (c). We also show how to use our prototype to generate a live video feed from the projector's-rather than the camera's-point of view, shown in (d). AbstractProgrammable coding of light between a source and a sensor has led to several important results in computational illumination, imaging and display. Little is known, however, about how to utilize energy most effectively, especially for applications in live imaging. In this paper, we derive a novel framework to maximize energy efficiency by "homogeneous matrix factorization" that respects the physical constraints of many coding mechanisms (DMDs/LCDs, lasers, etc.). We demonstrate energy-efficient imaging using two prototypes based on DMD and laser illumination. For our DMDbased prototype, we use fast local optimization to derive codes that yield brighter images with fewer artifacts in many transport probing tasks. Our second prototype uses a novel combination of a lowpower laser projector and a rolling shutter camera. We use this prototype to demonstrate never-seen-before capabilities such as (1) capturing live structured-light video of very bright scenes-even a light bulb that has been turned on; (2) capturing epipolar-only and indirect-only live video with optimal energy efficiency; (3) using a low-power projector to reconstruct 3D objects in challenging conditions such as strong indirect light, strong ambient light, and smoke; and (4) recording live video from a projector's-rather than the camera's-point of view.

Information-Driven Adaptive Structured-Light Scanners

“…In the field of structured-light reconstruction, several studies have suggested adaptive scanners (see for example [8,19,20,37]), and energy-efficient designs [24]. However, unlike previous attempts that observed specific image features and addressed a specific pattern decoding technique, we show how given a generative model for the sensing process we can obtain an adaptive scanner for various tasks, forming a decision-theoretic purposive [22] 3D scanner.…”

Snapshot difference imaging using correlation time-of-flight sensors

“…We would like to explore more sophisticated temporal coding strategies that may be able to separate direct and global illumination based on their temporal characteristics rather than their polarisation properties. We would also like to explore spatio-temporal coding strategies that would allow the light sources to be used as temporally-coded projectors rather than isotropic emitters [O'Toole et al 2015]. Finally, we would like to extend the application to mitigating multi-path interference for time-of-flight cameras and other tasks that may benefit from gradient cameras, such as Visual SLAM [Kim et al 2016] and 3D scanning [Matsuda et al 2015].…”

Depth Estimation Using Structured Light Flow — Analysis of Projected Pattern Flow on an Object’s Surface

“…Although photometric stereo can recover surface normals, they cannot recover absolute 3D distances; thus, their applicability is limited. The structured light technique has been used for practical applications [16,22,12]. Two primary approaches to encode positional information into patterns are temporal and spatial encoding.…”