Light-Field Image Super-Resolution Using Convolutional Neural Network

Yoon, Young-Jin; Jeon, Hae‐Gon; Yoo, Donggeun; Lee, Joon-Young; Kweon, In So

doi:10.1109/lsp.2017.2669333

Cited by 166 publications

(130 citation statements)

References 12 publications

Supporting

Mentioning

122

Contrasting

Order By: Relevance

“…Learning based methods. Recently, machine learning techniques have been applied to a variety of light field imaging applications such as super-resolution [41,40], novel view generation [19], single image to a light field image conversion [28], and material recognition [35].…”

Section: Related Workmentioning

confidence: 99%

EPINET: A Fully-Convolutional Neural Network Using Epipolar Geometry for Depth from Light Field Images

Shin

Jeon²,

Yoon³

et al. 2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

Self Cite

255

303

View full text Add to dashboard Cite

Light field cameras capture both the spatial and the angular properties of light rays in space. Due to its property, one can compute the depth from light fields in uncontrolled lighting environments, which is a big advantage over active sensing devices. Depth computed from light fields can be used for many applications including 3D modelling and refocusing. However, light field images from hand-held cameras have very narrow baselines with noise, making the depth estimation difficult. Many approaches have been proposed to overcome these limitations for the light field depth estimation, but there is a clear trade-off between the accuracy and the speed in these methods. In this paper, we introduce a fast and accurate light field depth estimation method based on a fully-convolutional neural network. Our network is designed by considering the light field geometry and we also overcome the lack of training data by proposing light field specific data augmentation methods. We achieved the top rank in the HCI 4D Light Field Benchmark on most metrics, and we also demonstrate the effectiveness of the proposed method on real-world light-field images.

show abstract

Section: Related Workmentioning

confidence: 99%

EPINET: A Fully-Convolutional Neural Network Using Epipolar Geometry for Depth from Light Field Images

Shin

Jeon²,

Yoon³

et al. 2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

Self Cite

255

303

View full text Add to dashboard Cite

show abstract

“…The output of the spatial CNN is then fed into a second CNN to perform angular super-resolution. While the approach in [10] takes at the input of the spatial CNN pairs or 4-tuples of neighboring views, leading to three spatial CNNs to be learned, a single CNN is proposed by the same authors in [11] to process each view independently. The problem of angular super-resolution of light fields is also addressed in [12] using an architecture based on two CNNs, one CNN being used to estimate disparity maps and the second CNN being used to synthesis intermediate views.…”

Section: Introductionmentioning

confidence: 99%

“…In this paper, we propose a spatial light field superresolution method using a deep CNN (DCNN) with ten convolutional layers. Instead of using DCNN to restore each sub-aperture image independently, as done in [10], [11], we restore all sub-aperture images within a light field simultaneously. This allows us to exploit both spatial and angular information to restore the light field and thus generate light fields which are angularly coherent.…”

Section: Introductionmentioning

confidence: 99%

Light Field Super-Resolution using a Low-Rank Prior and Deep Convolutional Neural Networks

Farrugia

Guillemot

2019

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Light field imaging has recently known a regain of interest due to the availability of practical light field capturing systems that offer a wide range of applications in the field of computer vision. However, capturing high-resolution light fields remains technologically challenging since the increase in angular resolution is often accompanied by a significant reduction in spatial resolution. This paper describes a learning-based spatial light field super-resolution method that allows the restoration of the entire light field with consistency across all sub-aperture images. The algorithm first uses optical flow to align the light field and then reduces its angular dimension using low-rank approximation. We then consider the linearly independent columns of the resulting low-rank model as an embedding, which is restored using a deep convolutional neural network (DCNN). The super-resolved embedding is then used to reconstruct the remaining sub-aperture images. The original disparities are restored using inverse warping where missing pixels are approximated using a novel light field inpainting algorithm. Experimental results show that the proposed method outperforms existing light field super-resolution algorithms, achieving PSNR gains of 0.23 dB over the second best performing method. This performance can be further improved using iterative back-projection as a post-processing step.

show abstract

“…With a large training dataset, CNN proves to be able to compete with BM3D. CNN also proves its efficiency in capturing the spatial-angular structures of the LF in applications such as LF super-resolution [25] and view synthesis [26], [27]. In this work, we aim at designing an LF denoiser utilizing the CNN's capacities in capturing LF parallax details from noisy observations.…”

Section: Introductionmentioning

confidence: 99%

Light Field Denoising via Anisotropic Parallax Analysis in a CNN Framework

Chen

Hou

Chau

2018

IEEE Signal Process. Lett.

View full text Add to dashboard Cite

Light field (LF) cameras provide perspective information of scenes by taking directional measurements of the focusing light rays. The raw outputs are usually dark with additive camera noise, which impedes subsequent processing and applications. We propose a novel LF denoising framework based on anisotropic parallax analysis (APA). Two convolutional neural networks are jointly designed for the task: first, the structural parallax synthesis network predicts the parallax details for the entire LF based on a set of anisotropic parallax features. These novel features can efficiently capture the high frequency perspective components of a LF from noisy observations. Second, the view-dependent detail compensation network restores non-Lambertian variation to each LF view by involving view-specific spatial energies. Extensive experiments show that the proposed APA LF denoiser provides a much better denoising performance than state-of-the-art methods in terms of visual quality and in preservation of parallax details.Index Terms-Light field, anisotropic parallax feature, denoising, convolutional neural networks J. Chen, and L.-P. Chau are with the

show abstract

Light-Field Image Super-Resolution Using Convolutional Neural Network

Cited by 166 publications

References 12 publications

EPINET: A Fully-Convolutional Neural Network Using Epipolar Geometry for Depth from Light Field Images

EPINET: A Fully-Convolutional Neural Network Using Epipolar Geometry for Depth from Light Field Images

Light Field Super-Resolution using a Low-Rank Prior and Deep Convolutional Neural Networks

Light Field Denoising via Anisotropic Parallax Analysis in a CNN Framework

Contact Info

Product

Resources

About