An Integrated CNN-based Post Processing Filter For Intra Frame in Versatile Video Coding

Wang, Mingze; Wan, Shuai; Gong, Hao; Yu, Yuanfang; Liu, Yang

doi:10.1109/apsipaasc47483.2019.9023240

“…• QP-specific training: dedicating one model for each QP or a range of QPs [8,66]. • QP-map training: providing QP as an input to the network [12,27,33,50,57]. Each approach has benefits and drawbacks.…”

Section: Methods Based On Coding Informationmentioning

confidence: 99%

“…Except for QP, which has a key influence on the distortion level, the use of other coding information is mostly overlooked in the existing studies. To further improve this aspect, in some more advanced works, coding information such as partitioning, prediction and residual information are also used [10][11][12]. However, these approaches are mainly applied to intra coded frames.…”

Section: Introductionmentioning

confidence: 99%

A CNN-Based Prediction-Aware Quality Enhancement Framework for VVC

Nasiri

¹

,

Hamidouche

²

,

Morin

³

et al. 2021

IEEE Open J. Signal Process.

View full text Add to dashboard Cite

This paper presents a framework for Convolutional Neural Network (CNN)-based quality enhancement task, by taking advantage of coding information in the compressed video signal. The motivation is that normative decisions made by the encoder can significantly impact the type and strength of artifacts in the decoded images. In this paper, the main focus has been put on decisions defining the prediction signal in intra and inter frames. This information has been used in the training phase as well as input to help the process of learning artifacts that are specific to each coding type. Furthermore, to retain a low memory requirement for the proposed method, one model is used for all Quantization Parameters (QPs) with a QP-map, which is also shared between luma and chroma components. In addition to the Post Processing (PP) approach, the In-Loop Filtering (ILF) codec integration has also been considered, where the characteristics of the Group of Pictures (GoP) are taken into account to boost the performance. The proposed CNN-based Quality Enhancement (QE) framework has been implemented on top of the Versatile Video Coding (VVC) Test Model (VTM-10). Experiments show that the prediction-aware aspect of the proposed method improves the coding efficiency gain of the default CNN-based QE method by 1.52%, in terms of BD-BR, at the same network complexity compared to the default CNN-based QE filter.

show abstract

“…• QP-specific training: dedicating one model for each QP or a range of QPs [8,66]. • QP-map training: providing QP as an input to the network [12,27,33,50,57]. Each approach has benefits and drawbacks.…”

Section: Methods Based On Coding Informationmentioning

confidence: 99%

Prediction-Aware Quality Enhancement of VVC Using CNN

Nasiri

¹

,

Hamidouche

²

,

Morin

³

et al. 2020

2020 IEEE International Conference on Visual Communications and Image Processing (VCIP)

View full text Add to dashboard Cite

This paper presents a framework for Convolutional Neural Network (CNN)-based quality enhancement task, by taking advantage of coding information in the compressed video signal. The motivation is that normative decisions made by the encoder can significantly impact the type and strength of artifacts in the decoded images. In this paper, the main focus has been put on decisions defining the prediction signal in intra and inter frames. This information has been used in the training phase as well as input to help the process of learning artifacts that are specific to each coding type. Furthermore, to retain a low memory requirement for the proposed method, one model is used for all Quantization Parameters (QPs) with a QP-map, which is also shared between luma and chroma components. In addition to the Post Processing (PP) approach, the In-Loop Filtering (ILF) codec integration has also been considered, where the characteristics of the Group of Pictures (GoP) are taken into account to boost the performance. The proposed CNN-based Quality Enhancement (QE) framework has been implemented on top of the Versatile Video Coding (VVC) Test Model (VTM-10). Experiments show that the prediction-aware aspect of the proposed method improves the coding efficiency gain of the default CNN-based QE method by 1.52%, in terms of BD-BR, at the same network complexity compared to the default CNN-based QE filter.

show abstract

“…The method presented in [ 28 ] exhibited a significant improvement in perceptual visual quality, achieving a reduction of 2.8% in performance on the BD-BR for the RA configuration compared to VVC/H.266. Wang et al [ 29 ] aimed to enhance the visual quality of decoded images by incorporating partitioning information with QP information, introducing a three-branch network. The method described in [ 29 ] demonstrated a notable improvement in perceptual visual quality, achieving a reduction of 6.5% in performance on the BD-BR for the AI configuration compared to VVC/H.266.…”

Section: Related Workmentioning

confidence: 99%

Block Partitioning Information-Based CNN Post-Filtering for EVC Baseline Profile

Choi

2024

Sensors

0

View full text Add to dashboard Cite

The need for efficient video coding technology is more important than ever in the current scenario where video applications are increasing worldwide, and Internet of Things (IoT) devices are becoming widespread. In this context, it is necessary to carefully review the recently completed MPEG-5 Essential Video Coding (EVC) standard because the EVC Baseline profile is customized to meet the specific requirements needed to process IoT video data in terms of low complexity. Nevertheless, the EVC Baseline profile has a notable disadvantage. Since it is a codec composed only of simple tools developed over 20 years, it tends to represent numerous coding artifacts. In particular, the presence of blocking artifacts at the block boundary is regarded as a critical issue that must be addressed. To address this, this paper proposes a post-filter using a block partitioning information-based Convolutional Neural Network (CNN). The proposed method in the experimental results objectively shows an approximately 0.57 dB for All-Intra (AI) and 0.37 dB for Low-Delay (LD) improvements in each configuration by the proposed method when compared to the pre-post-filter video, and the enhanced PSNR results in an overall bitrate reduction of 11.62% for AI and 10.91% for LD in the Luma and Chroma components, respectively. Due to the huge improvement in the PSNR, the proposed method significantly improved the visual quality subjectively, particularly in blocking artifacts at the coding block boundary.

show abstract

An Integrated CNN-based Post Processing Filter For Intra Frame in Versatile Video Coding

Cited by 9 publications

References 12 publications

A CNN-Based Prediction-Aware Quality Enhancement Framework for VVC

A CNN-Based Prediction-Aware Quality Enhancement Framework for VVC

Prediction-Aware Quality Enhancement of VVC Using CNN

Block Partitioning Information-Based CNN Post-Filtering for EVC Baseline Profile

Contact Info

Product

Resources

About