Sensor saturation in Fourier multiplexed imaging

Abstract

Optically multiplexed image acquisition techniques have become increasingly popular for encoding different exposures, color channels, light fields, and other properties of light onto two-dimensional image sensors. Recently, Fourier-based multiplexing and reconstruction approaches have been introduced in order to achieve a superior light transmission of the employed modulators and better signalto-noise characteristics of the reconstructed data.We show in this paper that Fourier-based reconstruction approaches suffer from severe artifacts in the case of sensor saturation, i.e. when the dynamic range of the scene exceeds the capabilities of the image sensor. We analyze the problem, and propose a novel combined optical light modulation and computational reconstruction method that not only suppresses such artifacts, but also allows us to recover a wider dynamic range than existing image-space multiplexing approaches.

A theory of plenoptic multiplexing

“…In particular, spatial patterns are chosen such that the different slices of the plenoptic function are encoded into different frequency bands. In computer graphics, this optical heterodyne approach has so far been used for capturing light fields [32,31], occluder information [14], and high dynamic range color photographs [34]. Spatially encoded light fields were recently analyzed in Fourier space and it was demonstrated that Fourier reconstruction algorithms apply as well [10].…”

Gradient domain color restoration of clipped highlights

“…Unlike this approach and later approaches (e.g. [12,18,8,23]), we do not aim to change the photography process to increase the amount of information captured about a scene, but instead aim to extract as much information as possible from a single, given photograph. LDR to HDR enhancement: Reconstructing an HDR image from a single exposure with clipped values is a challenging problem that yields only approximate solutions based on heuristics or manual user intervention [17,2,22,7].…”

An overview of computational photography

“…Mitsunaga et al [101] performed high dynamic range imaging by estimating the sensor response first and performing specific modifications, while Tumblin et al [45] designed a sensor recording image gradient, and Wetzstein et al [46] inserted plug-in filters (e.g. graduated neutral density filters) in front of the lens or sensor.…”

“…1) Depth can be recovered from defocus analysis, because the depth of field is closely related to the distance. The typical approaches include introducing coded aperture patterns [46,[143][144][145] or multiple apertures [114], computing from the image pairs captured using different aperture sizes [146][147][148]33]. Levin [149] compares the performances of different aperture codes in depth estimation and gives a mathematical analysis of the results using a geometrical optics model.…”

“…capturing the light field [2,6,32], removing veiling glare [34,39], extending the dynamic range [35][36][37], and increasing the field of view [38,40]. Some other implementations of computational sensors include the introduction of sensor motion to extend the depth of field [41,42] or perform motion deblurring [43], and building sensing patterns for image super resolution [44] or high dynamic range imaging [45,46] 3) Computational illumination. Computational illumination usually controls the photographic illumination in a structured way, to create new images that meet specific demands by introducing some computational strategies.…”