Near-invariant blur for depth and 2D motion via time-varying light field analysis

Abstract

Recently, several camera designs have been proposed for either making defocus blur invariant to scene depth or making motion blur invariant to object motion. The benefit of such invariant capture is that no depth or motion estimation is required to remove the resultant spatially uniform blur. So far, the techniques have been studied separately for defocus and motion blur, and object motion has been assumed to be 1D (e.g., horizontal). This paper explores a more general capture method that makes both defocus blur and motion blur nearly invariant to scene depth and in-plane 2D object motion. We formulate the problem as capturing a time-varying light field through a time-varying light field modulator at the lens aperture, and perform 5D (4D light field + 1D time) analysis of all the existing computational cameras for defocus/motion-only deblurring and their hybrids. This leads to a surprising conclusion that focus sweep, previously known as a depth-invariant capture method that moves the plane of focus through a range of scene depth during exposure, is near-optimal both in terms of depth and 2D motion-invariance and in terms of high frequency preservation for certain combinations of depth and motion ranges. Using our prototype camera, we demonstrate joint defocus and motion deblurring for moving scenes with depth variation.

Detecting planar surface using a light-field camera with application to distinguishing real scenes from printed photos

“…This is known as the Epipolar Plane Image [13] or the light-field [14]. Light-fields have attracted a lot of attention in recent years [15,16,17,18].…”

Point spread function and depth-invariant focal sweep point spread function for plenoptic camera 20

“…+∞ −∞ =    (19) Using h(x , y , x 0 , y 0 ) to represent the PSF of the whole system, Eq. (19) can be rewritten as:…”

“…For the conventional cameras, FSPSF is obtained by changing the focal plane during the exposure time, which is carried out by moving the sensor [18]. Y. Bando et al proved the depth-invariant property of FSPSF theoretically for the conventional cameras [19] and S. Kuthirummal et al proved both the space and depth invariant properties of FSPSF empirically [20]. R. Yokoya et al extended the FSPSF to catadioptric imaging system and verified the depth-invariant property [21].…”

Computationally efficient background subtraction in the light field domain

“…8 Light-fields have attracted a lot of attention in recent years. [12][13][14][15] We can capture EPIs using either a linear camera array or equivalently (and more practically) by assuming that the scene is static and linearly moving a single camera (Figure 1). This latter setup is more practical since a plenoptic function (more precisely a slice of it) can be captured easily and efficiently using a handheld device's internal camera.…”