Estimation of Signal Distortion Using Effective Sampling Density for Light Field-Based Free Viewpoint Video

Abstract

In a light field-based free viewpoint video (LF-based FVV) system, effective sampling density (ESD) is defined as the number of rays per unit area of the scene that has been acquired and is selected in the rendering process for reconstructing an unknown ray. This paper extends the concept of ESD and shows that ESD is a tractable metric that quantifies the joint impact of the imperfections of LF acquisition and rendering. By deriving and analyzing ESD for the commonly used LF acquisition and rendering methods, it is shown that ESD is an effective indicator determined by system parameters and can be used to directly estimate output video distortion without access to the ground truth. This claim is verified by extensive numerical simulations and comparison to PSNR. Furthermore, an empirical relationship between the output distortion (in PSNR) and the calculated ESD is established to allow direct assessment of the overall video distortion without an actual implementation of the system. A small scale subjective user study is also conducted which indicates a correlation of 0.91 between ESD and perceived quality. (ESD) is defined as the number of rays per unit area of the scene that has been acquired and is selected in the rendering process for reconstructing an unknown ray. This paper extends the concept of ESD and shows that ESD is a tractable metric that quantifies the joint impact of the imperfections of LF acquisition and rendering. By deriving and analyzing ESD for the commonly used LF acquisition and rendering methods, it is shown that ESD is an effective indicator determined by system parameters and can be used to directly estimate output video distortion without access to the ground truth. This claim is verified by extensive numerical simulations and comparison to PSNR. Furthermore, an empirical relationship between the output distortion (in PSNR) and the calculated ESD is established to allow direct assessment of the overall video distortion without an actual implementation of the system. A small scale subjective user study is also conducted which indicates a correlation of 0.91 between ESD and perceived quality.

Light‐field head‐mounted displays reduce the visual effort: A user study

“…The participants had to position a three‐dimensional (3D) object in three dimensions; however, the study investigated the effects of different input modalities, but not the effects of the light‐field technology itself. Also, different light‐field rendering effects on user acceptance were evaluated in look‐and‐feel studies 34 …”

A Light Field Sparse and Reconstruction Framework for Improving Rendering Quality

“…Light field sampling is closely related to the reflection of the object on the light rays. Therefore, plenoptic sampling has emerged by analyzing light transport characteristics [27]- [29]. Additionally, Zhu et al used the autocorrelation theorem to study the frequency estimation of the plenoptic function.…”

No-Reference Light Field Image Quality Assessment Based on Spatial-Angular Measurement

“…Although there exist the evaluation methods for light field rendering and visualization [47], [48], few objective LFI quality evaluation methods have been proposed. Specifically, Fang et al proposed a FR LFI-QA method that measures the gradient magnitude similarity of reference and distorted EPIs [49].…”