Learning a Deep Convolutional Network for Light-Field Image Super-Resolution

Yoon, Young-Jin; Jeon, Hae‐Gon; Yoo, Donggeun; Lee, Joon-Young; Kweon, In So

doi:10.1109/iccvw.2015.17

Cited by 297 publications

(220 citation statements)

References 27 publications

Supporting

Mentioning

218

Contrasting

Order By: Relevance

“…Average PSNR gains of 3.7dB, 4.2dB and 2.1dB on the HCI, Stanford and INRIA datasets respectively were obtained for spatial super-resolution using BM+PCA+RR compared to the recent DCNN scheme [9] which obtained the second best results in our experiments. Moreover, subjective results clearly show that the proposed angular super-resolution method provides sharper synthesized intermediate sub-aperture images when compared to those obtained using the DCNN based schemes [9], [11].…”

Section: Introductionmentioning

confidence: 78%

“…Hence, solving this ill-posed inverse problem requires introducing some priors on V H , which can be a statistical prior such as a GMM model [8], or priors learned from training data as in [9] and in our proposed method. show the actual projection of the low-quality patch volume p L j onto the low-quality sub-space using E L , followed by the projection from the low-to high-quality subspaces using the projection Φ and then the projection from the high-quality subspace to reconstruct the high-quality patch volumep H j .…”

Section: Proposed Methods Let Vmentioning

confidence: 99%

“…Experimental results with both synthetic light fields (from the HCI dataset) and real light fields (from the Stanford and the INRIA datasets) show that the proposed method outperforms state-of-the-art single-image super-resolution methods [13], [14], [15] applied on each sub-aperture image separately, and significantly outperfoms prior light field super-resolution algorithms for both spatial [8], [9] and angular [9], [11] super-resolution. Average PSNR gains of 3.7dB, 4.2dB and 2.1dB on the HCI, Stanford and INRIA datasets respectively were obtained for spatial super-resolution using BM+PCA+RR compared to the recent DCNN scheme [9] which obtained the second best results in our experiments.…”

Section: Introductionmentioning

confidence: 96%

“…The high resolution 4D-patches are estimated using a linear minimum mean square error (LMMSE) estimator, assuming a disparity-dependent Gaussian Mixture Model (GMM) for the patch structure. More recently, the authors in [9] adopt a deep convolutional neural network (DCNN) to perform both spatial and angular super-resolution. This method first employs a spatial DCNN similar to [10] followed by an angular DCNN which synthesizes intermediate views.…”

Section: Introductionmentioning

confidence: 99%

“…The DCNN scheme presented in [9] models the angular super-resolution function based on the available neighbouring sub-aperture images independently from the viewpoint being approximated. Instead, we model the angular super-resolution problem using multi-linear models, i.e.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Super Resolution of Light Field Images Using Linear Subspace Projection of Patch-Volumes

Farrugia

Galea

Guillemot

2017

IEEE J. Sel. Top. Signal Process.

View full text Add to dashboard Cite

Abstract-Light field imaging has emerged as a very promising technology in the field of computational photography. Cameras are becoming commercially available for capturing real-world light fields. However, capturing high spatial resolution light fields remains technologically challenging, and the images rendered from real light fields have today a significantly lower spatial resolution compared to traditional 2D cameras. This paper describes an example-based super-resolution algorithm for light fields, which allows the increase of the spatial resolution of the different views in a consistent manner across all sub-aperture images of the light field. The algorithm learns linear projections between subspaces of reduced dimension in which reside patch-volumes extracted from the light field. The method is extended to cope with angular super-resolution, where 2D patches of intermediate sub-aperture images are approximated from neighbouring subaperture images using multivariate ridge regression. Experimental results show significant quality improvement when compared to state-of-the-art single-image super-resolution methods applied on each view separately, as well as when compared to a recent light field super-resolution techniques based on deep learning.

show abstract

Section: Introductionmentioning

confidence: 78%

Section: Proposed Methods Let Vmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 96%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Super Resolution of Light Field Images Using Linear Subspace Projection of Patch-Volumes

Farrugia

Galea

Guillemot

2017

IEEE J. Sel. Top. Signal Process.

View full text Add to dashboard Cite

show abstract

Deep‐learning‐based harmonization and super‐resolution of near‐surface air temperature from CMIP6 models (1850–2100)

Wei

Wang

Feng

et al. 2022

Intl Journal of Climatology

View full text Add to dashboard Cite

Future global temperature change will have significant effects on society and ecosystems. Earth system models (ESM) are the primary tools to explore future climate change. However, ESMs have great uncertainty and often run at a coarse spatial resolution (usually about 2°). Accurate high‐spatial‐resolution temperature dataset are needed to improve our understanding of temperature variations and for many other applications. We apply Super resolution (SR) in computer vision using deep learning (DL) to merge 31 ESMs data. The proposed method performs data merging, bias‐correction, and spatial downscaling simultaneously. The CRU TS (Climate Research Unit gridded Time Series) data is used as reference data in the model training process. To find a suitable DL method, we select five SR methodologies with different structures. We compare the performances of the methods based on mean square error (MSE), mean absolute error (MAE) and Pearson correlation coefficient (R). The best method is used to merge the projected monthly data (1850–1900), and monthly future scenarios data (2015–2100), at the high spatial resolution of 0.5°. Results show that the merged data have considerably improved performance compared with individual ESM data and their ensemble mean (EM), both spatially and temporally. The MAE shows significant improvement; the spatial distribution of the MAE widens along the latitudes in the Northern Hemisphere. The MAE of merged data is ranging from 0.60 to 1.50, the South American (SA) has the lowest error and the Europe has the highest error. The merged product has excellent performance when the observation data is smooth with few fluctuations in the time series. This work demonstrates the applicability and effectiveness of the DL methods in data merging, bias‐correction and spatial downscaling when enough training data are available. Data can be accessed at https://doi.org/10.5281/zenodo.5746632.

show abstract

Simultaneoussuperresolutionreconstruction and distortion correction for single‐shotEPI DWIusing deep learning

Wang

et al. 2023

Magnetic Resonance in Med

View full text Add to dashboard Cite

Single-shot (SS) EPI is widely used for clinical DWI. This study aims to develop an end-to-end deep learning-based method with a novel loss function in an improved network structure to simultaneously increase the resolution and correct distortions for SS-EPI DWI. Theory and Methods: Point-spread-function (PSF)-encoded EPI can provide high-resolution, distortion-free DWI images. A distorted image from SS-EPI can be described as the convolution between a PSF function with a distortion-free image. The deconvolution process to recover the distortion-free image can be achieved with a convolution neural network, which also learns the mapping function between low-resolution SS-EPI and high-resolution reference PSF-EPI to achieve superresolution. To suppress the oversmoothing effect, we proposed a modified generative adversarial network structure, in which a dense net with gradient map guidance and a multilevel fusion block was used as the generator. A fractional anisotropy loss was proposed to utilize the diffusion anisotropy information among diffusion directions. In vivo brain DWI data were used to test the proposed method. Results: The results show that distortion-corrected high-resolution DWI images with restored structural details can be obtained from low-resolution SS-EPI images by taking advantage of the high-resolution anatomical images. Additionally, the proposed network can improve the quantitative accuracy of diffusion metrics compared with previously reported networks. Conclusion:Using high-resolution, distortion-free EPI-DWI images as references, a deep learning-based method to simultaneously increase the perceived resolution and correct distortions for low-resolution SS-EPI was proposed. The results show that DWI image quality and diffusion metrics can be improved.

show abstract

Learning a Deep Convolutional Network for Light-Field Image Super-Resolution

Cited by 297 publications

References 27 publications

Super Resolution of Light Field Images Using Linear Subspace Projection of Patch-Volumes

Super Resolution of Light Field Images Using Linear Subspace Projection of Patch-Volumes

Deep‐learning‐based harmonization and super‐resolution of near‐surface air temperature from CMIP6 models (1850–2100)

Simultaneoussuperresolutionreconstruction and distortion correction for single‐shotEPI DWIusing deep learning

Contact Info

Product

Resources

About