A quantitative approach for comparison and evaluation of light field rendering techniques

2015

IEEE Trans. Multimedia

Self Cite

In a light field-based free viewpoint video (LF-based FVV) system, effective sampling density (ESD) is defined as the number of rays per unit area of the scene that has been acquired and is selected in the rendering process for reconstructing an unknown ray. This paper extends the concept of ESD and shows that ESD is a tractable metric that quantifies the joint impact of the imperfections of LF acquisition and rendering. By deriving and analyzing ESD for the commonly used LF acquisition and rendering methods, it is shown that ESD is an effective indicator determined by system parameters and can be used to directly estimate output video distortion without access to the ground truth. This claim is verified by extensive numerical simulations and comparison to PSNR. Furthermore, an empirical relationship between the output distortion (in PSNR) and the calculated ESD is established to allow direct assessment of the overall video distortion without an actual implementation of the system. A small scale subjective user study is also conducted which indicates a correlation of 0.91 between ESD and perceived quality. (ESD) is defined as the number of rays per unit area of the scene that has been acquired and is selected in the rendering process for reconstructing an unknown ray. This paper extends the concept of ESD and shows that ESD is a tractable metric that quantifies the joint impact of the imperfections of LF acquisition and rendering. By deriving and analyzing ESD for the commonly used LF acquisition and rendering methods, it is shown that ESD is an effective indicator determined by system parameters and can be used to directly estimate output video distortion without access to the ground truth. This claim is verified by extensive numerical simulations and comparison to PSNR. Furthermore, an empirical relationship between the output distortion (in PSNR) and the calculated ESD is established to allow direct assessment of the overall video distortion without an actual implementation of the system. A small scale subjective user study is also conducted which indicates a correlation of 0.91 between ESD and perceived quality.

Section: Theoretical and Simulation Resultsmentioning

confidence: 99%

Estimation of Signal Distortion Using Effective Sampling Density for Light Field-Based Free Viewpoint Video

2015

IEEE Trans. Multimedia

Self Cite

“…To address this, a FVV simulation system [22] was utilized. The simulator takes a 3D model of a scene and generates both reference cameras images and ground truth images.…”

Section: Experimental Validationmentioning

confidence: 99%

Non-uniform sampling of plenoptic signal based on the scene complexity variations for a free viewpoint video system

2013 IEEE International Conference on Image Processing

Zamani-Farahani

et al. 2013

Self Cite

Acquisition of a free viewpoint video (FVV) system is theoretically considered as a problem of plenoptic signal sampling. It is typically performed by using a regular camera grid. While a regular acquisition itself results in non-uniform sampling density, this non-uniformity does not match the scene complexity and frequency variations. This paper proposes an irregular acquisition method for optimum non-uniform plenoptic sampling corresponding to the variations of the scene complexity. Specifically, scene complexity is measured through analyzing DCT coefficients of reference images of the scene, describing the frequency behavior of the plenoptic signal over the scene space. An optimization model is formulated to calculate the optimum configurations of the acquisition cameras including positions and orientations. The theoretical analysis and numerical simulations demonstrate that the rendered video quality can be significantly improved (around 40% in mean PSNR) by employing the proposed irregular acquisition compared with the regular camera grid.

“…To address this, a simulation system proposed in [30][31][32] was utilized. The simulator takes a 3D model of a scene and generates both reference cameras images and ground truth images.…”

Section: Experimental Validationmentioning

confidence: 99%

Optimization of the number of rays in interpolation for light field based free viewpoint systems

2015 IEEE International Conference on Multimedia and Expo (ICME)

2015

Self Cite

Light field (LF) rendering is widely used in free viewpoint video systems (FVV). Different methods have been proposed to employ depth maps to improve the rendering quality. However, estimation of depth is often error-prone. In this paper, a new method based on the concept of effective sampling density (ESD) is proposed for evaluating the depth-based LF rendering algorithms at different levels of errors in the depth estimation. In addition, for a given rendering quality, we provide an estimation of number of rays required in the interpolation algorithm to compensate for the adverse effect caused by errors in depth maps. The proposed method is particularly useful in designing a rendering algorithm with inaccurate knowledge of depth to achieve the required rendering quality. Both the theoretical study and numerical simulations have verified the efficacy of the proposed method. OPTIMIZATION OF THE NUMBER OF RAYS IN INTERPOLATION FOR LIGHT FIELD BASED FREE VIEWPOINT SYSTEMS Hooman Shidanshidi, Farzad Safaei, Wanqing LiICT Research Institute, University of Wollongong, Australia hooman@uow.edu.au, farzad@uow.edu, wanqing@uow.edu.au ABSTRACT Light field (LF) rendering is widely used in free viewpoint video systems (FVV). Different methods have been proposed to employ depth maps to improve the rendering quality. However, estimation of depth is often error-prone. In this paper, a new method based on the concept of effective sampling density (ESD) is proposed for evaluating the depth-based LF rendering algorithms at different levels of errors in the depth estimation. In addition, for a given rendering quality, we provide an estimation of number of rays required in the interpolation algorithm to compensate for the adverse effect caused by errors in depth maps. The proposed method is particularly useful in designing a rendering algorithm with inaccurate knowledge of depth to achieve the required rendering quality. Both the theoretical study and numerical simulations have verified the efficacy of the proposed method.