Light Field Super-Resolution via Adaptive Feature Remixing

Ko, Keunsoo; Koh, Yeong Jun; Chang, Seunghyuk; Kim, Chang-Su

doi:10.1109/tip.2021.3069291

Cited by 53 publications

(35 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the end, a color predictor network aggregates the information from the warped views to generate the final result. Adaptive feature remixing-based spatial and angular resolution enhancement using two dedicated CNNs is presented in [76], which yields high-quality super-resolved images regardless of the angular coordinates of input view images. In [40], the authors applied a 2D CNN on epipolar images to reconstruct novel views of the light-field.…”

Section: B Deep Learning-based Methodsmentioning

confidence: 99%

Attention Mechanism-Based Light-Field View Synthesis

et al. 2022

View full text Add to dashboard Cite

The angular information of light lost in conventional images but preserved and stored in light-fields plays an instrumental role in many applications such as depth estimation, 3D reconstruction and post-capture refocusing. However, the limited angular resolution of light-field images due to the consumer hardware limitations is a major drawback in its widespread adoption. In this article, we present a novel deep learning-based light-field view synthesis method from a sparse set of input views. Our proposed method, an end-to-end trainable network, utilizes convolutional block attention modules to enhance the built-in depth image-based rendering. The proposed convolutional block attention module consists of two attention modules sequentially applied in the channel and spatial dimensions to focus the network on critical features. The proposed network architecture is a combination of three sub-networks, one for stereo feature extraction, another for disparity estimation, and the last for attention-based refinement. We present two schemes for the refinement network that perform equally well but differ in the number of parameters by 1.2% and execution time by 44%. Quantitative and qualitative results on four challenging real-world datasets show the superiority of the proposed method. Our proposed method shows considerable PSNR gains, around 1 dB compared to the state of the art and around 0.5 dB over our previous method LFVS-AM. In addition, ablation studies demonstrate the effectiveness of each module of the proposed method. Finally, the parallax edge precision-recall curve shows that our method better preserves parallax details.

show abstract

Section: B Deep Learning-based Methodsmentioning

confidence: 99%

Attention Mechanism-Based Light-Field View Synthesis

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Both disparity and confidence maps were estimated in their method for image warping and view synthesis. Ko et al [58] proposed an adaptive feature remixing approach for spatial and angular SR. In their method, feature of each view was integrated with the ones from adjacent views based on the estimated disparity.…”

Section: Angular Super-resolutionmentioning

confidence: 99%

Disentangling Light Fields for Super-Resolution and Disparity Estimation

Wang,

et al. 2022

Preprint

View full text Add to dashboard Cite

Light field (LF) cameras record both intensity and directions of light rays, and encode 3D scenes into 4D LF images. Recently, many convolutional neural networks (CNNs) have been proposed for various LF image processing tasks. However, it is challenging for CNNs to effectively process LF images since the spatial and angular information are highly inter-twined with varying disparities. In this paper, we propose a generic mechanism to disentangle these coupled information for LF image processing. Specifically, we first design a class of domain-specific convolutions to disentangle LFs from different dimensions, and then leverage these disentangled features by designing task-specific modules. Our disentangling mechanism can well incorporate the LF structure prior and effectively handle 4D LF data. Based on the proposed mechanism, we develop three networks (i.e., DistgSSR, DistgASR and DistgDisp) for spatial super-resolution, angular super-resolution and disparity estimation. Experimental results show that our networks achieve state-of-the-art performance on all these three tasks, which demonstrates the effectiveness, efficiency, and generality of our disentangling mechanism. Project page: https://yingqianwang.github.io/DistgLF/.

show abstract

“…These parametric curves produce promising results, but it is challenging to find reliable parameters, which are effective for diverse images. Recently, with the success of convolutional neural networks (CNNs) in the field of low-level vision [11]- [14], CNN-based CE methods also have been proposed, yielding outstanding performance [15].…”

Section: Introductionmentioning

confidence: 99%

IceNet for Interactive Contrast Enhancement

2021

IEEE Access

Self Cite

View full text Add to dashboard Cite

A CNN-based interactive contrast enhancement algorithm, called IceNet, is proposed in this paper, which enables a user to adjust image contrast easily according to his or her preference. Specifically, a user provides a parameter for controlling the global brightness and two types of scribbles to darken or brighten local regions in an image. Then, given these annotations, IceNet estimates a gamma map for the pixel-wise gamma correction. Finally, through color restoration, an enhanced image is obtained. The user may provide annotations iteratively to obtain a satisfactory image. IceNet is also capable of producing a personalized enhanced image automatically, which can serve as a basis for further adjustment if so desired. Moreover, to train IceNet effectively and reliably, we propose three differentiable losses. Extensive experiments demonstrate that IceNet can provide users with satisfactorily enhanced images.INDEX TERMS Interactive contrast enhancement, personalized contrast enhancement, convolutional neural network, and adaptive gamma correction.

show abstract

Light Field Super-Resolution via Adaptive Feature Remixing

Cited by 53 publications

References 35 publications

Attention Mechanism-Based Light-Field View Synthesis

Attention Mechanism-Based Light-Field View Synthesis

Disentangling Light Fields for Super-Resolution and Disparity Estimation

IceNet for Interactive Contrast Enhancement

Contact Info

Product

Resources

About