CNN-based approach for visual quality improvement on HEVC

Lee, Young-Woon; Kim, Ji-Hae; Choi, Young-Ju; Kim, Byung-gyu

doi:10.1109/icce.2018.8326088

Cited by 8 publications

(3 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…But the bottom-up decision causes unnecessary calculation when a CTU is non-split or split into only a few large CUs in Kim et al adopted CNN to predict split or nonsplit for CU depth decision both inter and intra-coding in the HEVC [18]. Lee et al have improved visual quality for HEVC using CNN [19]. They constructed a little simple network model for intra prediction mode.…”

Section: Deep Learning Based Approachmentioning

confidence: 99%

CNN-based Fast Split Mode Decision Algorithm for Versatile Video Coding (VVC) Inter Prediction

Yeo¹,

Kim²

2021

J Multimed Inf Syst

Self Cite

View full text Add to dashboard Cite

Versatile Video Coding (VVC) is the latest video coding standard developed by Joint Video Exploration Team (JVET). In VVC, the quadtree plus multi-type tree (QT+MTT) structure of coding unit (CU) partition is adopted, and its computational complexity is considerably high due to the brute-force search for recursive rate-distortion (RD) optimization. In this paper, we aim to reduce the time complexity of inter-picture prediction mode since the inter prediction accounts for a large portion of the total encoding time. The problem can be defined as classifying the split mode of each CU. To classify the split mode effectively, a novel convolutional neural network (CNN) called multi-level tree (MLT-CNN) architecture is introduced. For boosting classification performance, we utilize additional information including inter-picture information while training the CNN. The overall algorithm including the MLT-CNN inference process is implemented on VVC Test Model (VTM) 11.0. The CUs of size 128×128 can be the inputs of the CNN. The sequences are encoded at the random access (RA) configuration with five QP values {22, 27, 32, 37, 42}. The experimental results show that the proposed algorithm can reduce the computational complexity by 11.53% on average, and 26.14% for the maximum with an average 1.01% of the increase in Bjøntegaard delta bit rate (BDBR). Especially, the proposed method shows higher performance on the sequences of the A and B classes, reducing 9.81%~26.14% of encoding time with 0.95%~3.28% of the BDBR increase.

show abstract

Section: Deep Learning Based Approachmentioning

confidence: 99%

CNN-based Fast Split Mode Decision Algorithm for Versatile Video Coding (VVC) Inter Prediction

Yeo¹,

Kim²

2021

J Multimed Inf Syst

Self Cite

View full text Add to dashboard Cite

show abstract

“…Different from most of the aforementioned approaches based on linear computation, neural networks show their mighty potential in video coding due to their violent non-linear fitting capability. Typically, the neural network-based approaches are implemented into the video compression architecture to strengthen the coding performance of each specific section, such as mode estimation [11,12], partitioning [13,14], intra prediction [15][16][17][18][19][20][21], inter prediction [22] and post-processing [23,24].…”

Section: Introductionmentioning

confidence: 99%

A GAN-Based Video Intra Coding

Zhong

Wang

et al. 2021

Electronics

View full text Add to dashboard Cite

Intra prediction is a vital part of the image/video coding framework, which is designed to remove spatial redundancy within a picture. Based on a set of predefined linear combinations, traditional intra prediction cannot cope with coding blocks with irregular textures. To tackle this drawback, in this article, we propose a Generative Adversarial Network (GAN)-based intra prediction approach to enhance intra prediction accuracy. Specifically, with the superior non-linear fitting ability, the well-trained generator of GAN acts as a mapping from the adjacent reconstructed signals to the prediction unit, implemented into both encoder and decoder. Simulation results show that for All-Intra configuration, our proposed algorithm achieves, on average, a 1.6% BD-rate cutback for luminance components compared with video coding reference software HM-16.15 and outperforms previous similar works.

show abstract

“…Thanks to the rapid development of flexible vision sensors and visual sensor networks, computer vision has entered a new phase. The improvements in computer vision-derived applications (such as vehicle tracking [1], facial interactions [2] and age assessment [3]) and various coding standards [4] also feed back the development of vision sensors. Vision sensors need the support of underlying .…”

Section: Introductionmentioning

confidence: 99%

Mask-Refined R-CNN: A Network for Refining Object Details in Instance Segmentation

Zhang

Chu

Leng

et al. 2020

Sensors

152

View full text Add to dashboard Cite

With the rapid development of flexible vision sensors and visual sensor networks, computer vision tasks, such as object detection and tracking, are entering a new phase. Accordingly, the more challenging comprehensive task, including instance segmentation, can develop rapidly. Most state-of-the-art network frameworks, for instance, segmentation, are based on Mask R-CNN (mask region-convolutional neural network). However, the experimental results confirm that Mask R-CNN does not always successfully predict instance details. The scale-invariant fully convolutional network structure of Mask R-CNN ignores the difference in spatial information between receptive fields of different sizes. A large-scale receptive field focuses more on detailed information, whereas a small-scale receptive field focuses more on semantic information. So the network cannot consider the relationship between the pixels at the object edge, and these pixels will be misclassified. To overcome this problem, Mask-Refined R-CNN (MR R-CNN) is proposed, in which the stride of ROIAlign (region of interest align) is adjusted. In addition, the original fully convolutional layer is replaced with a new semantic segmentation layer that realizes feature fusion by constructing a feature pyramid network and summing the forward and backward transmissions of feature maps of the same resolution. The segmentation accuracy is substantially improved by combining the feature layers that focus on the global and detailed information. The experimental results on the COCO (Common Objects in Context) and Cityscapes datasets demonstrate that the segmentation accuracy of MR R-CNN is about 2% higher than that of Mask R-CNN using the same backbone. The average precision of large instances reaches 56.6%, which is higher than those of all state-of-the-art methods. In addition, the proposed method requires low time cost and is easily implemented. The experiments on the Cityscapes dataset also prove that the proposed method has great generalization ability.

show abstract

CNN-based approach for visual quality improvement on HEVC

Cited by 8 publications

References 5 publications

CNN-based Fast Split Mode Decision Algorithm for Versatile Video Coding (VVC) Inter Prediction

CNN-based Fast Split Mode Decision Algorithm for Versatile Video Coding (VVC) Inter Prediction

A GAN-Based Video Intra Coding

Mask-Refined R-CNN: A Network for Refining Object Details in Instance Segmentation

Contact Info

Product

Resources

About