Non-refractive modulators for encoding and capturing scene appearance and depth

Abstract

We analyze the modulation of a light field via non-refracting attenuators. In the most general case, any desired modulation can be achieved with attenuators having four degrees of freedom in ray-space. We motivate the discussion with a universal 4D ray modulator (ray-filter) which can attenuate the intensity of each ray independently. We describe operating of such a fantasy rayfilter in the context of altering the 4D light field incident on a 2D camera sensor. Ray-filters are difficult to realize in practice but we can achieve reversible encoding for light field capture using patterned attenuating mask. Two mask-based designs are analyzed in this framework. The first design closely mimics the angle-dependent ray-sorting possible with the ray filter. The second design [17] exploits frequency-domain modulation to achieve a more efficient encoding. We extend these designs for optimal sampling of light field by matching the modulation function to the specific shape of the band-limit frequency transform of light field. We also show how a hand-held version of an attenuator based light field camera can be built using a medium-format digital camera and an inexpensive mask. CVPR 2008This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research Laboratories, Inc.; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Research Laboratories, Inc. All rights reserved.

A theory of plenoptic multiplexing

“…In particular, spatial patterns are chosen such that the different slices of the plenoptic function are encoded into different frequency bands. In computer graphics, this optical heterodyne approach has so far been used for capturing light fields [32,31], occluder information [14], and high dynamic range color photographs [34]. Spatially encoded light fields were recently analyzed in Fourier space and it was demonstrated that Fourier reconstruction algorithms apply as well [10].…”

Light-Field Microscopy with a Consumer Light-Field Camera

“…A modern elaboration of this approach where the sensor and a micro-lens array are combined to form an incamera light-field imaging system is the Hand-Held Plenoptic Camera [17]. Alternatively, sensor masks [22,21] or a light pipe [15] can be arranged such that in-camera lightfields can be recorded. Other methods use external arrays of mirrors instead of lenses [7], or external lens arrays [4].…”

An overview of computational photography

“…Representative work includes: Veeraraghavan et al [6,72], who designed mask enhanced cameras to heterodyne the band-limited light fields in the frequency domain for refocusing; Ng [32], who analyzed the 4D light field captured by a lenslet based camera [12] in the Fourier domain and selected its specific slice trajectory for refocusing; and Georgiev et al [73], who analyzed the light field cameras under a unified framework in the frequency domain. Other researchers performed demosaicing [74] or computed 3D occluders and cast shadows [75] in the frequency domain.…”