CNN-Based Driving of Block Partitioning for Intra Slices Encoding

Galpin, Franck; Racapé, Fabien; Jaiswal, Sunil Prasad; Bordes, Philippe; Léannec, Fabrice Le; François, Édouard

doi:10.1109/dcc.2019.00024

Cited by 37 publications

(35 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Subsequently, Galpin et al [16] suggested a scheme deciding the CU partition directly by predicting all possible CU boundaries between adjacent 4×4 blocks using ResNet model [17]. But the bottom-up decision causes unnecessary calculation when a CTU is non-split or split into only a few large CUs in Kim et al adopted CNN to predict split or nonsplit for CU depth decision both inter and intra-coding in the HEVC [18].…”

Section: Deep Learning Based Approachmentioning

confidence: 99%

CNN-based Fast Split Mode Decision Algorithm for Versatile Video Coding (VVC) Inter Prediction

Yeo¹,

Kim²

2021

J Multimed Inf Syst

View full text Add to dashboard Cite

Versatile Video Coding (VVC) is the latest video coding standard developed by Joint Video Exploration Team (JVET). In VVC, the quadtree plus multi-type tree (QT+MTT) structure of coding unit (CU) partition is adopted, and its computational complexity is considerably high due to the brute-force search for recursive rate-distortion (RD) optimization. In this paper, we aim to reduce the time complexity of inter-picture prediction mode since the inter prediction accounts for a large portion of the total encoding time. The problem can be defined as classifying the split mode of each CU. To classify the split mode effectively, a novel convolutional neural network (CNN) called multi-level tree (MLT-CNN) architecture is introduced. For boosting classification performance, we utilize additional information including inter-picture information while training the CNN. The overall algorithm including the MLT-CNN inference process is implemented on VVC Test Model (VTM) 11.0. The CUs of size 128×128 can be the inputs of the CNN. The sequences are encoded at the random access (RA) configuration with five QP values {22, 27, 32, 37, 42}. The experimental results show that the proposed algorithm can reduce the computational complexity by 11.53% on average, and 26.14% for the maximum with an average 1.01% of the increase in Bjøntegaard delta bit rate (BDBR). Especially, the proposed method shows higher performance on the sequences of the A and B classes, reducing 9.81%~26.14% of encoding time with 0.95%~3.28% of the BDBR increase.

show abstract

Section: Deep Learning Based Approachmentioning

confidence: 99%

CNN-based Fast Split Mode Decision Algorithm for Versatile Video Coding (VVC) Inter Prediction

Yeo¹,

Kim²

2021

J Multimed Inf Syst

View full text Add to dashboard Cite

show abstract

“…Also for JEM, Jin [11] selects out of five classes of quad-and binary-tree depth ranges for 32×32 blocks using the hinge loss. For JEM and VVC, Galpin [12] and Tissier [13] output split probabilities at boundaries of all 4×4 blocks in a 64×64 input block and select the tested partitionings based on them. Kuanar [14] classifies HEVC CTUs depending on their texture to choose between five sets of predetermined CU depths and prediction modes.…”

Section: Related Workmentioning

confidence: 99%

CNN-based parameter selection for fast VVC intra-picture encoding

Tech

Pfaff

Schwarz

et al. 2021

2021 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

This paper presents two new methods for fast VVC intra-picture encoding. Both are based on an approach that uses a CNN for blockadaptive parameter estimation. The parameters restrict the multitype-tree (MTT) partitionings tested by the encoder. The methods aim for an improvement of the approach by further constraints with additional parameters. Adding parameters increases the time required for training data generation exponentially. This raises the question which parameters to add and how. To explore further partitioning restrictions, the first method adds parameters controlling the block sizes the MTT can start from. Although this leads to four parameters, we can exploit that some of their combinations are invalid. To investigate whether testing fewer prediction and transform modes is feasible, the second method adds a single parameter that restricts their number jointly. The paper evaluates hypothetical and actual encoding time reductions for VTM-10.2. The first method outperforms our other and other existing method: The encoding time decreases by 50% with a bit rate increase of 0.7%.

show abstract

“…A lightweight and adjustable QTBT partition scheme based on ML technology is presented in [27], which achieves an adjustable compromise between reduced complexity and reduced video quality in H.266/VVC. A technical overview of a deeplearning-based method aiming at optimizing next- generation hybrid video encoders is provided in [28] for driving the CU partition. A CNN oriented fast QTBT partition decision scheme is introduced in [29] for intercoding.…”

Section: Related Workmentioning

confidence: 99%

Adaptive CU Split Decision Based on Deep Learning and Multifeature Fusion for H.266/VVC

Zhao

Wang

Zhang

2020

Scientific Programming

View full text Add to dashboard Cite

With the development of technology, the hardware requirement and expectations of user for visual enjoyment are getting higher and higher. The multitype tree (MTT) architecture is proposed by the Joint Video Experts Team (JVET). Therefore, it is necessary to determine not only coding unit (CU) depth but also its split mode in the H.266/Versatile Video Coding (H.266/VVC). Although H.266/VVC achieves significant coding performance on the basis of H.265/High Efficiency Video Coding (H.265/HEVC), it causes significantly coding complexity and increases coding time, where the most time-consuming part is traversal calculation rate-distortion (RD) of CU. To solve these problems, this paper proposes an adaptive CU split decision method based on deep learning and multifeature fusion. Firstly, we develop a texture classification model based on threshold to recognize complex and homogeneous CU. Secondly, if the complex CUs belong to edge CU, a Convolutional Neural Network (CNN) structure based on multifeature fusion is utilized to classify CU. Otherwise, an adaptive CNN structure is used to classify CUs. Finally, the division of CU is determined by the trained network and the parameters of CU. When the complex CUs are split, the above two CNN schemes can successfully process the training samples and terminate the rate-distortion optimization (RDO) calculation for some CUs. The experimental results indicate that the proposed method reduces the computational complexity and saves 39.39% encoding time, thereby achieving fast encoding in H.266/VVC.

show abstract

CNN-Based Driving of Block Partitioning for Intra Slices Encoding

Cited by 37 publications

References 5 publications

CNN-based Fast Split Mode Decision Algorithm for Versatile Video Coding (VVC) Inter Prediction

CNN-based Fast Split Mode Decision Algorithm for Versatile Video Coding (VVC) Inter Prediction

CNN-based parameter selection for fast VVC intra-picture encoding

Adaptive CU Split Decision Based on Deep Learning and Multifeature Fusion for H.266/VVC

Contact Info

Product

Resources

About