Optimal CTU-Level Rate Control Model for HEVC Based on Deep Convolutional Features

Abstract

This paper proposes an optimal rate control model based on deep neural network (DNN) features to improve the coding tree unit (CTU)-level rate control in high-efficiency video coding for conversational videos. The proposed algorithm extracts high-level features from the original and previously reconstructed CTU blocks based on a predefined DNN model of the visual geometry group (VGG-16) network. Then, the correlation of the high-level feature and quantization parameter (QP) values of previously coded CTUs is explored for subjective visual characteristics to estimate the CTU-level rate control model parameters (alpha and beta) and the bit allocation of each CTU. Therefore, this paper also proposes a new model for Lambda estimation for each CTU by improving its relationship with the estimated bits per pixel to control the rate and relative distortion. Furthermore, the Lambda and QP boundary settings were adjusted based on the proposed perceptual model to ensure the rate control accuracy of each CTU. The results of experiments with the proposed algorithm, when compared to the rate control model in HM-16.20, reveal higher bitrate accuracy and an average BD-rate gain based on PSNR, SSIM, and MSSSIM metrics using the low-delay-P configuration. INDEX TERMS Deep neural network, high efficiency video coding (HEVC), rate control, video coding

Deep Convolutional Feature-Driven Rate Control for the HEVC Encoders

“…The original studies [2,3] claimed that these values were provided based on video content. However, studies have discovered [30][31][32] that the existing R-λ model in the HM software is not optimal for the coding structure for the RA configuration.…”

“…This work determines the values of α and β for the framelevel rate control by considering the high-level features of a particular convolutional layer of the VGG-16 architecture. From previous work in [30], it is observed that a strong interrelationship between α-λ and β-λ is vital for improving the quality of the bpp-λ relationship. Consequently, it will also result in increasing the reconstructed video quality subjectively.…”

State-Of-The-Art In Video Processing: Compression, Optimization And Retrieval

“…Whereas the positive value indicates that the bit rate is amplified for thealike PSNR value. CTU rate control [12] estimates the allocation of bits to each CTU by exploiting the correlation between quantization parameters and features. Deep neural network based model improves the efficiency of CTU level rate control video coding.…”