Effects of Different Full-Reference Quality Assessment Metrics in End-to-End Deep Video Coding

Abstract

Visual quality assessment is often used as a key performance indicator (KPI) to evaluate the performance of electronic devices. There exists a significant association between visual quality assessment and electronic devices. In this paper, we bring attention to alternative choices of perceptual loss function for end-to-end deep video coding (E2E-DVC), which can be used to reduce the amount of data generated by electronic sensors and other sources. Thus, we analyze the effects of different full-reference quality assessment (FR-QA) metrics on E2E-DVC. First, we select five optimization-suitable FR-QA metrics as perceptual objectives, which are differentiable and thus support back propagation, and use them to optimize an E2E-DVC model. Second, we analyze the rate–distortion (R-D) behaviors of an E2E-DVC model under different loss function optimizations. Third, we carry out subjective human perceptual tests on the reconstructed videos to show the performance of different FR-QA optimizations on subjective visual quality. This study reveals the effects of the competing FR-QA metrics on E2E-DVC and provides a guide for further future study on E2E-DVC in terms of perceptual loss function design.

FLAME-VQA: A Fuzzy Logic-Based Model for High Frame Rate Video Quality Assessment

“…The findings highlighted the importance of screen size in perceived video quality and the limitations of existing objective quality models in capturing the effect of screen size changes. The issue of video compression and its relation to user perception is also studied in [30].…”

“…For the medium compression cluster associated with the VIDEO CRF input variable, we observed that the standard Gauss2 function did not fully align with the underlying data As in Figure 2, the x-axes values in Figure 3 correspond with the properties of the video sequences discussed in Section 3.1. Specifically, there were six different video frame rate values (24,30,60,82,98, and 120 fps), with CRF values ranging from 0 to 63, depending on the fps and the resulting bitrate. To compute the SI and TI values, we utilized the MSU Quality Measurement Tool (version 14.1) [43], calculating them for each of the 480 test sequences and then multiplying them by 1000 to enable clustering.…”