Enhancing Quality for HEVC Compressed Videos

Abstract

The latest High Efficiency Video Coding (HEVC) standard has been increasingly applied to generate video streams over the Internet. However, HEVC compressed videos may incur severe quality degradation, particularly at low bit-rates. Thus, it is necessary to enhance the visual quality of HEVC videos at the decoder side. To this end, this paper proposes a Quality Enhancement Convolutional Neural Network (QE-CNN) method that does not require any modification of the encoder to achieve quality enhancement for HEVC. In particular, our QE-CNN method learns QE-CNN-I and QE-CNN-P models to reduce the distortion of HEVC I and P/B frames, respectively. The proposed method differs from the existing CNN-based quality enhancement approaches, which only handle intra-coding distortion and are thus not suitable for P/B frames. Our experimental results validate that our QE-CNN method is effective in enhancing quality for both I and P/B frames of HEVC videos. To apply our QE-CNN method in time-constrained scenarios, we further propose a Time-constrained Quality Enhancement Optimization (TQEO) scheme. Our TQEO scheme controls the computational time of QE-CNN to meet a target, meanwhile maximizing the quality enhancement. Next, the experimental results demonstrate the effectiveness of our TQEO scheme from the aspects of time control accuracy and quality enhancement under different time constraints. Finally, we design a prototype to implement our TQEO scheme in a real-time scenario.

Partition-Aware Adaptive Switching Neural Networks for Post-Processing in HEVC

“…Recently, due to the impressive achievements of deep neural networks in computer vision and image processing tasks, many deep learning based methods are proposed to further reduce the visual artifacts of decoded images [15]- [18], [20]. More specifically, Dong et al [15] developed an Artifacts Reduction CNN (ARCNN) built upon [24], which successfully reduces the JPEG compression artifacts.…”

“…Similar to images, there are also a lot of works focusing on deep learning based artifact reduction for decoded videos [17], [18], [20]. Park and Kim [17] proposed an In-loop Filter CNN (IFCNN) based on SRCNN [24] to replace the Sample Adaptive Offset (SAO) filter in HEVC.…”

“…However, since these works do not consider any prior information and only take decoded frame as input, they have limitations in handling complex degradation introduced by HEVC. Recently, Yang et al [20] proposed QECNN that has specific architectures for HEVC intra-and inter-coding frames. This amounts to process I and P/B frames in HEVC with two separate models.…”

“…Compared with the existing compressed video postprocessing methods [18], [20], our network has two differences: (1) We introduce two stream inputs to include both the decoded frame and the partition information. (2) We use residual architecture to perform the feature extraction.…”

“…As a result, the visual artifacts of HEVC-compressed videos can be more effectively reduced under the same bit rate. Moreover, the recent study [20] attempts to use two CNNs with different architectures to handle degradation variation in HEVC intra and inter coding. Though it has the specific architectures for HEVC intra-and inter-coding frames, the degradation variation within decoded frames (e.g., among patches) is neglected.…”