LightGAN: A Deep Generative Model for Light Field Reconstruction

Meng, Nan; Zhang, Ge; Zeng, Tianjiao; Lam, Edmund Y.

doi:10.1109/access.2020.3004477

Cited by 19 publications

(8 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Farrugia [75] embeds lowrank priors into the deep convolutional network to restore the consistency of the entire light field on all sub-aperture images. Meng [56] merged the high-dimensional convolutional layer for the special structure of light field images into GAN [76] to find the correlation between adjacent light field images.…”

Section: Inter-image-similarity-based Lfsrmentioning

confidence: 99%

“…Wu [27] and Yuan [54] et al used EPI for super-resolution processing of the light field. With the prosperity of deep learning, many novel network structures have also been applied to the super-resolution processing of light field images, such as Zhang [24] used residual networks, Zhu [55] combined CNN with long short term memory (LSTM), Meng [56] used generative adversarial network (GAN). At present, the research on the super-resolution of light field images is not satisfied with higher super-resolution quality, however, has expanded to how to improve the processing speed of super-resolution while ensuring the super-resolution quality, such as Wang [57], Ma [58].…”

mentioning

confidence: 99%

See 1 more Smart Citation

Reivew of Light Field Image Super-Resolution

Song

et al. 2022

Electronics

View full text Add to dashboard Cite

Currently, light fields play important roles in industry, including in 3D mapping, virtual reality and other fields. However, as a kind of high-latitude data, light field images are difficult to acquire and store. Thus, the study of light field super-resolution is of great importance. Compared with traditional 2D planar images, 4D light field images contain information from different angles in the scene, and thus the super-resolution of light field images needs to be performed not only in the spatial domain but also in the angular domain. In the early days of light field super-resolution research, many solutions for 2D image super-resolution, such as Gaussian models and sparse representations, were also used in light field super-resolution. With the development of deep learning, light field image super-resolution solutions based on deep-learning techniques are becoming increasingly common and are gradually replacing traditional methods. In this paper, the current research on super-resolution light field images, including traditional methods and deep-learning-based methods, are outlined and discussed separately. This paper also lists publicly available datasets and compares the performance of various methods on these datasets as well as analyses the importance of light field super-resolution research and its future development.

show abstract

Section: Inter-image-similarity-based Lfsrmentioning

confidence: 99%

mentioning

confidence: 99%

Reivew of Light Field Image Super-Resolution

Song

et al. 2022

Electronics

View full text Add to dashboard Cite

show abstract

“…Through the proposed adversarial loss and content loss, SRGAN could recover the fine texture details of super-resolved images, and infer photo-realistic natural images. Similarly, considering such benefits, Meng et al [27] also incorporate the high-dimensional convolution layers into the GAN framework to learn the high correlations among the neighboring light field views. By doing so, their model could generate novel views images with good fidelity and clearer edges.…”

Section: Related Workmentioning

confidence: 99%

Fast Light Field Angular Resolution Enhancement Using Convolutional Neural Network

et al. 2021

View full text Add to dashboard Cite

Current light field angular super-resolution algorithm based on deep learning has excessive computation cost and low operational efficiency, for sequentially up-sampling on each lenslet region of the light field image. In this paper, we propose a novel convolutional neural network to fastly enhance the angular resolution, via wholesale up-sampling lenslet regions. Firstly, the network simultaneously extracts the angular information of all lenslet regions on the input light field image. Then, from the extracted angular information, four feature maps are predicted. Especially, the angular resolution of each feature map is the same as that of the input light field image. Finally, to enhance the angular resolution, we integrate four feature maps into one image, by referring to angular information arrangement in lenslet regions. The experimental results verify the effectiveness of our proposed method. We only need 11.95s to enhance(actually double) the angular resolution of one light field image with 2562×3724 pixels, which surpasses 20 times faster than the state-ofthe-art method. Meanwhile, our method also achieves average PSNR gains of 0.39 dB. INDEX TERMS Light-field(LF), angular super-resolution, convolutional neural network(CNN).

show abstract

“…To reconstruct the high-frequency spatial details [46], [47], we also add the content perceptual loss component c given by…”

Section: F Loss Functionmentioning

confidence: 99%

Light Field View Synthesis via Aperture Disparity and Warping Confidence Map

Meng,

Li,

Liu

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

This paper presents a learning-based approach to synthesize the view from an arbitrary camera position given a sparse set of images. A key challenge for this novel view synthesis arises from the reconstruction process, when the views from different input images may not be consistent due to obstruction in the light path. We overcome this by jointly modeling the epipolar property and occlusion in designing a convolutional neural network. We start by defining and computing an aperture flow map, which approximates the parallax and measures the pixel-wise shift between two views. While this relates to freespace rendering and can fail near the object boundaries, we further develop a propagation confidence map to address pixel occlusion in these challenging regions. The proposed method is evaluated on diverse real-world and synthetic light field scenes, and it shows outstanding performance over several state-of-theart techniques.

show abstract

LightGAN: A Deep Generative Model for Light Field Reconstruction

Cited by 19 publications

References 45 publications

Reivew of Light Field Image Super-Resolution

Reivew of Light Field Image Super-Resolution

Fast Light Field Angular Resolution Enhancement Using Convolutional Neural Network

Light Field View Synthesis via Aperture Disparity and Warping Confidence Map

Contact Info

Product

Resources

About