Objective Quality Assessment in Free-Viewpoint Video Production

Abstract

This paper addresses the problem of objectively quantifying accuracy in free-viewpoint video production. Freeviewpoint video makes use of geometric scene reconstruction and renders novel views using the appearance sampled in multiple camera images. Previous work typically adopts an objective evaluation of geometric accuracy against ground truth data or a subjective evaluation of visual quality in view synthesis. We consider two production scenarios, human performance capture in a highly constrained studio environment and sports production in a large scale external environment. The accuracy of scene reconstruction is typically limited and absolute geometric accuracy does not necessarily reflect the quality of free-viewpoint rendering. A framework is introduced to quantify error at the point of view synthesis. The approach can be applied as a full-reference metric to measure fidelity to a ground truth image or as a no-reference metric to measure the error in rendering. The framework is applied to a data-set with known geometric accuracy and a comparison is presented for studio-based and sports production scenarios.

3DTV view generation using uncalibrated pure rotating and zooming cameras

“…We use no-reference (no ground truth) evaluation method proposed in [31] to measure the error in registering scene appearance in image-based rendering. Two new view images at the center (ratio 50:50) between the two reference cameras are rendered.…”

“…21. Two metrics d 90 [31] and peak signal to noise ratio (PSNR) are computed over the overlapping pixels to measure the registration error in these new view images (reprojected appearances). d 90 tells us about the overall distance of misaligned pixels between two images.…”

Real-time free viewpoint video from uncalibrated cameras using plane-sweep algorithm

“…View at ground-truth camera is then synthesized to measure visual errors. Two metrics d 90 proposed in [20] and PSNR (Peak Signal to Noise Ratio) are computed to measure the errors in the synthesized images. d 90 tells us about the overall distance of misaligned pixels between synthesized image and ground-truth reference.…”

Objective quality analysis for free-viewpoint DIBR

“…For measuring the rendering quality of each building block, we employ the PSNR metric. Other objective quality metrics exist, such as described in [11,12,13]. These methods try to correlate the measurements with the perceived quality.…”