Compressive light field photography using overcomplete dictionaries and optimized projections

Abstract

Figure 1: Light field reconstruction from a single coded projection. We explore sparse reconstructions of 4D light fields from optimized 2D projections using light field atoms as the fundamental building blocks of natural light fields. This example shows a coded sensor image captured with our camera prototype (upper left), and the recovered 4D light field (lower left and center). Parallax is successfully recovered (center insets) and allows for post-capture refocus (right). Even complex lighting effects, such as occlusion, specularity, and refraction, can be recovered, being exhibited by the background, dragon, and tiger, respectively. AbstractLight field photography has gained a significant research interest in the last two decades; today, commercial light field cameras are widely available. Nevertheless, most existing acquisition approaches either multiplex a low-resolution light field into a single 2D sensor image or require multiple photographs to be taken for acquiring a high-resolution light field. We propose a compressive light field camera architecture that allows for higher-resolution light fields to be recovered than previously possible from a single image. The proposed architecture comprises three key components: light field atoms as a sparse representation of natural light fields, an optical design that allows for capturing optimized 2D light field projections, and robust sparse reconstruction methods to recover a 4D light field from a single coded 2D projection. In addition, we demonstrate a variety of other applications for light field atoms and sparse coding techniques, including 4D light field compression and denoising.

Enhancing the performance of the light field microscope using wavefront coding

“…These might include using different types of phase masks at neighboring microlenses, or replacing the microlens array with an alternative sensing element, such as the coded attenuation mask proposed in [25] for capturing compressive light fields in a camera setup.…”

Towards Motion Aware Light Field Video for Dynamic Scenes

“…LF capture: Existing LF cameras can be divided into two main categories: (a) single shot [24,13,30,20,27,12,22], and (b) multiple shot [18,23,3]. Most single shot light field cameras multiplex the 4-D LF onto the 2D sensor, losing spatial resolution to capture the angular information in the LF.…”

“…Such cameras employ either a lenslet array close to the sensor [24,12], a mask close to the sensor [30] or an array of lens/prism outside the main lens [13]. Recently, [22] extended the mask based method of [30] to exploit sparse representations in order to recover full resolution LF. Our method is similar in spirit but works to recover the loss Figure 4.…”

“…The concept of placing a coded mask close to the sensor for LF capture was proposed by [30]. Coded masks have also been used for estimating scene depth from single image [15],and for compressive LF [22] and video acquisition [21].…”

Overlapped Fourier coding for optical aberration removal

“…Although not used by the current OFC setup, phase-based modulation may also lead to accurate sample phase recovery [15][16][17]. Furthermore, an SLM can help acquire an incoherent light field near a camera's native spatial resolution [18,19]. Resolution-improved incoherent intensity images may also be obtained via SLM-generated structured illumination [20][21][22].…”