Light Field Blind Motion Deblurring

Srinivasan, Pratul P.; Ng, Ren; Ramamoorthi, Ravi

doi:10.1109/cvpr.2017.253

Cited by 44 publications

(56 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…See [19], [50] for detailed overviews of LF imaging. To date, LF image processing has been applied to a variety of applications including image-based rendering [9], [22], [23], post-capture image refocus [13], [30], SfM [20], lens aberration correction [18], spatial [3] and temporal [46] superresolution, video stabilization [37], motion deblurring [38], and depth imaging [24], [41], [43], [47], [49]. In this work, we explore robust LF feature detection and matching for improving applications in reconstruction including SfM.…”

Section: Related Workmentioning

confidence: 99%

“…Light field (LF) imaging is an established tool in computer vision offering advantages in computational complexity and robustness to challenging scenarios [7], [10], [29], [38], [48]. This is due both to a more favourable signal-to-noise ratio (SNR) / depth of field tradeoff than for conventional cameras, and to the rich depth, occlusion, and native non-Lambertian surface capture inherently supported by LFs.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

LiFF: Light Field Features in Scale and Depth

Dansereau

Girod

Wetzstein

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

Feature detectors and descriptors are key low-level vision tools that many higher-level tasks build on. Unfortunately these fail in the presence of challenging light transport effects including partial occlusion, low contrast, and reflective or refractive surfaces. Building on spatio-angular imaging modalities offered by emerging light field cameras, we introduce a new and computationally efficient 4D light field feature detector and descriptor: LiFF. LiFF is scale invariant and utilizes the full 4D light field to detect features that are robust to changes in perspective. This is particularly useful for structure from motion (SfM) and other tasks that match features across viewpoints of a scene. We demonstrate significantly improved 3D reconstructions via SfM when using LiFF instead of the leading 2D or 4D features, and show that LiFF runs an order of magnitude faster than the leading 4D approach. Finally, LiFF inherently estimates depth for each feature, opening a path for future research in light field-based SfM.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

LiFF: Light Field Features in Scale and Depth

Dansereau

Girod

Wetzstein

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

show abstract

“…Unlike [23], in this paper, we address the problem of blind deblurring which is a more highly ill-posed problem. Srinivasan et al [26] solved the light field deblurring under 3D camera motion path and showed visually pleasing result. However, their methods do not consider 3D orientation change of the camera.…”

Section: Related Workmentioning

confidence: 99%

“…Note that we use the camera path model used in [19,20]. However, the Bézier camera path model used in [26] can be directly applied to (7) as well. D tm (x, u) is also represented by D tr (x, c) by forward warping and interpolation.…”

Section: Motion Blur Formulation In Light Fieldmentioning

confidence: 99%

Joint Blind Motion Deblurring and Depth Estimation of Light Field

Lee

Park

et al. 2018

Computer Vision – ECCV 2018

View full text Add to dashboard Cite

Removing camera motion blur from a single light field is a challenging task since it is highly ill-posed inverse problem. The problem becomes even worse when blur kernel varies spatially due to scene depth variation and high-order camera motion. In this paper, we propose a novel algorithm to estimate all blur model variables jointly, including latent sub-aperture image, camera motion, and scene depth from the blurred 4D light field. Exploiting multi-view nature of a light field relieves the inverse property of the optimization by utilizing strong depth cues and multi-view blur observation. The proposed joint estimation achieves high quality light field deblurring and depth estimation simultaneously under arbitrary 6-DOF camera motion and unconstrained scene depth. Intensive experiment on real and synthetic blurred light field confirms that the proposed algorithm outperforms the state-of-the-art light field deblurring and depth estimation methods.

show abstract

“…OF THE PROPOSED DBLRNET. IT IS COMPOSED OF TWO CONVOLUTIONAL LAYERS (L1 AND L2), 14 RESIDUAL BLOCKS, TWO CONVOLUTIONAL LAYERS (L31 AND L32) WITHOUT SKIP CONNECTION, AND THREE ADDITIONAL CONVOLUTIONAL LAYERS (L33, L34 AND L35).EACH RESIDUAL BLOCK CONTAINS TWO CONVOLUTIONAL LAYERS, WHICH ARE INDICATED BY L(X) AND L(X+1) IN THE TABLE, WHERE "X"EQUALS 3,5,7,9,11,13,15,17,19,21,23,25, 27 AND 29 RESPECTIVELY FOR THESE RESIDUAL BLOCKS.…”

mentioning

confidence: 99%

Adversarial Spatio-Temporal Learning for Video Deblurring

Zhang

Luo

Zhong

et al. 2019

IEEE Trans. on Image Process.

149

View full text Add to dashboard Cite

Camera shake or target movement often leads to undesired blur effects in videos captured by a hand-held camera. Despite significant efforts having been devoted to video-deblur research, two major challenges remain: 1) how to model the spatio-temporal characteristics across both the spatial domain (i.e., image plane) and the temporal domain (i.e., neighboring frames) and 2) how to restore sharp image details with respect to the conventionally adopted metric of pixel-wise errors. In this paper, to address the first challenge, we propose a deblurring network (DBLRNet) for spatial-temporal learning by applying a 3D convolution to both the spatial and temporal domains. Our DBLRNet is able to capture jointly spatial and temporal information encoded in neighboring frames, which directly contributes to the improved video deblur performance. To tackle the second challenge, we leverage the developed DBLRNet as a generator in the generative adversarial network (GAN) architecture and employ a content loss in addition to an adversarial loss for efficient adversarial training. The developed network, which we name as deblurring GAN, is tested on two standard benchmarks and achieves the state-of-the-art performance.

show abstract

Light Field Blind Motion Deblurring

Cited by 44 publications

References 36 publications

LiFF: Light Field Features in Scale and Depth

LiFF: Light Field Features in Scale and Depth

Joint Blind Motion Deblurring and Depth Estimation of Light Field

Adversarial Spatio-Temporal Learning for Video Deblurring

Contact Info

Product

Resources

About