A Benchmark of Light Field View Interpolation Methods

Yue, Dingcheng; Gul, M. Shahzeb Khan; Bätz, Michel; Keinert, Joachim; Mantiuk, Rafał

doi:10.1109/icmew46912.2020.9106041

Cited by 10 publications

(6 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most works on NVS methods and NVS benchmarks [MST*20; DBD*23; MSO*19; WPYS21; FBD*19; LGZ*20; AAB23] evaluate on sparse hold‐out views using image quality metrics [FTS*23; ANA*20; MDC*21b; SB13; LAK*16]. An exception is the Light Field Benchmark [YKB*20], where light field interpolation methods were evaluated on video sequences. On the contrary, our focus is on assessing the perceptual quality of NVS methods and evaluating how well current objective metrics can predict subjective quality.…”

Section: Related Workmentioning

confidence: 99%

Perceptual Quality Assessment of NeRF and Neural View Synthesis Methods for Front‐Facing Views

Liang,

Wu,

Hanji

et al. 2024

Computer Graphics Forum

View full text Add to dashboard Cite

Neural view synthesis (NVS) is one of the most successful techniques for synthesizing free viewpoint videos, capable of achieving high fidelity from only a sparse set of captured images. This success has led to many variants of the techniques, each evaluated on a set of test views typically using image quality metrics such as PSNR, SSIM, or LPIPS. There has been a lack of research on how NVS methods perform with respect to perceived video quality. We present the first study on perceptual evaluation of NVS and NeRF variants. For this study, we collected two datasets of scenes captured in a controlled lab environment as well as in‐the‐wild. In contrast to existing datasets, these scenes come with reference video sequences, allowing us to test for temporal artifacts and subtle distortions that are easily overlooked when viewing only static images. We measured the quality of videos synthesized by several NVS methods in a well‐controlled perceptual quality assessment experiment as well as with many existing state‐of‐the‐art image/video quality metrics. We present a detailed analysis of the results and recommendations for dataset and metric selection for NVS evaluation.

show abstract

Section: Related Workmentioning

confidence: 99%

Perceptual Quality Assessment of NeRF and Neural View Synthesis Methods for Front‐Facing Views

Liang,

Wu,

Hanji

et al. 2024

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

“…These real world datasets assess performance under natural illumination and practical camera distortion. Also, we cross-validate the performance on a realworld gantry dataset [15].…”

Section: ) Datasetmentioning

confidence: 99%

“…As opposed to the previous evaluations, we will now crossvalidate the performance of the proposed method on a dataset captured using a gantry-based acquisition method [15]. Moreover, we also compared our results with the lightfield view synthesis approach LLFF [45] which is based on multi-plane images (MPI).…”

Section: B Cross-validation On Gantry Datasetmentioning

confidence: 99%

“…Apart from direct spatio-angular captures, several approaches have been proposed to increase the angular resolution of light-fields. Many conventional methods propose to reconstruct novel views using a small set of input views [15]- [18] with deep learning-based models outperform most of them. Recent advances in deep learning techniques for numerous computer vision and image processing problems, such as multi-view stereo [19], [20] and super-resolution [21], motivated the researchers to explore deep learningbased models for light-field view synthesis [22]- [24].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Attention Mechanism-Based Light-Field View Synthesis

et al. 2022

Self Cite

View full text Add to dashboard Cite

The angular information of light lost in conventional images but preserved and stored in light-fields plays an instrumental role in many applications such as depth estimation, 3D reconstruction and post-capture refocusing. However, the limited angular resolution of light-field images due to the consumer hardware limitations is a major drawback in its widespread adoption. In this article, we present a novel deep learning-based light-field view synthesis method from a sparse set of input views. Our proposed method, an end-to-end trainable network, utilizes convolutional block attention modules to enhance the built-in depth image-based rendering. The proposed convolutional block attention module consists of two attention modules sequentially applied in the channel and spatial dimensions to focus the network on critical features. The proposed network architecture is a combination of three sub-networks, one for stereo feature extraction, another for disparity estimation, and the last for attention-based refinement. We present two schemes for the refinement network that perform equally well but differ in the number of parameters by 1.2% and execution time by 44%. Quantitative and qualitative results on four challenging real-world datasets show the superiority of the proposed method. Our proposed method shows considerable PSNR gains, around 1 dB compared to the state of the art and around 0.5 dB over our previous method LFVS-AM. In addition, ablation studies demonstrate the effectiveness of each module of the proposed method. Finally, the parallax edge precision-recall curve shows that our method better preserves parallax details.

show abstract

“…The IBR methods synthesizing the novel views without explicitly modeling the scene depth [13,14], tend to fail with increasing baseline of the input views, whereas, depthbased methods can handle inputs with wider baseline (i.e. large disparities) [11,15].…”

Section: Introductionmentioning

confidence: 99%

Light-Field View Synthesis Using A Convolutional Block Attention Module

Gul¹,

Mukati

Bätz

et al. 2021

2021 IEEE International Conference on Image Processing (ICIP)

Self Cite

View full text Add to dashboard Cite

Consumer light-field (LF) cameras suffer from a low or limited resolution because of the angular-spatial trade-off. To alleviate this drawback, we propose a novel learning-based approach utilizing attention mechanism to synthesize novel views of a light-field image using a sparse set of input views (i.e., 4 corner views) from a camera array. In the proposed method, we divide the process into three stages, stereo-feature extraction, disparity estimation, and final image refinement. We use three sequential convolutional neural networks for each stage. A residual convolutional block attention module (CBAM) is employed for final adaptive image refinement. Attention modules are helpful in learning and focusing more on the important features of the image and are thus sequentially applied in the channel and spatial dimensions. Experimental results show the robustness of the proposed method. Our proposed network outperforms the stateof-the-art learning-based light-field view synthesis methods on two challenging real-world datasets by 0.5 dB on average. Furthermore, we provide an ablation study to substantiate our findings.

show abstract

A Benchmark of Light Field View Interpolation Methods

Cited by 10 publications

References 33 publications

Perceptual Quality Assessment of NeRF and Neural View Synthesis Methods for Front‐Facing Views

Perceptual Quality Assessment of NeRF and Neural View Synthesis Methods for Front‐Facing Views

Attention Mechanism-Based Light-Field View Synthesis

Light-Field View Synthesis Using A Convolutional Block Attention Module

Contact Info

Product

Resources

About