A CNN-Based Prediction-Aware Quality Enhancement Framework for VVC

Abstract

This paper presents a framework for Convolutional Neural Network (CNN)-based quality enhancement task, by taking advantage of coding information in the compressed video signal. The motivation is that normative decisions made by the encoder can significantly impact the type and strength of artifacts in the decoded images. In this paper, the main focus has been put on decisions defining the prediction signal in intra and inter frames. This information has been used in the training phase as well as input to help the process of learning artifacts that are specific to each coding type. Furthermore, to retain a low memory requirement for the proposed method, one model is used for all Quantization Parameters (QPs) with a QP-map, which is also shared between luma and chroma components. In addition to the Post Processing (PP) approach, the In-Loop Filtering (ILF) codec integration has also been considered, where the characteristics of the Group of Pictures (GoP) are taken into account to boost the performance. The proposed CNN-based Quality Enhancement (QE) framework has been implemented on top of the Versatile Video Coding (VVC) Test Model (VTM-10). Experiments show that the prediction-aware aspect of the proposed method improves the coding efficiency gain of the default CNN-based QE method by 1.52%, in terms of BD-BR, at the same network complexity compared to the default CNN-based QE filter.

CARNet:Compression Artifact Reduction for Point Cloud Attribute

“…Compression artifact reduction was extensively studied for restoring better reconstruction quality of compressed 2D images/videos, including both rules-based and learning-based methods, for either in-loop filtering or post-processing [10], [11], [14], [17], [18], [31]- [39]. Recently, learning-based solutions presented outstanding performance with remarkable quality improvement, which is mainly due to the use of powerful DNNs that effectively model spatial or spatiotemporal neighborhood characteristics for quality enhancement and artifact removal [12], [13].…”

“…Since the projection-based V-PCC adopts the HEVC or VVC as the compression backbone, rulesbased in-loop filters [10] including deblocking, sample adaptive offset (SAO), and/or adaptive loop filter (ALF) adopted in video codecs already significantly mitigate compression artifacts. Besides, recently-emerged learning-based filters can be augmented on top of existing codecs as either a postprocessing module [11], [12] or an in-loop function [13], [14] to further improve the quality of restored pixels (e.g., YUV or RGB). This work thus focuses on the compression artifact reduction (CAR) of G-PCC coded point cloud attribute (PCA).…”

Machine Learning for Multimedia Communications

“…For video compression, the benefits of learning-based solutions have been less immediate, mostly because temporal motion is still not accurately modeled by neural network architectures. Naturally, deep neural networks (DNNs) can be used to assist the complex encoding/decoding operations performed by the standard video coders, or to post-process the decoded video [ 51 , 52 ]. Entirely replacing the video coder with an end-to-end system is less straightforward.…”