Speed Up H.266/QTMT Intra-Coding Based on Predictions of ResNet and Random Forest Classifier

Huang, Yu-Huan; Chen, Jiann‐Jone; Tsai, Yao-Hong

doi:10.1109/icce50685.2021.9427626

Cited by 15 publications

(20 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Note that the R 𝑇 performances are not especially high for training and testing videos, such as B_BasketballDrive, C_RaceHorsesC, D_BasketballPass, and E_FourPeople, which justified the generalization capability of the learned model. Regarding the ACUCNN model, as compared to previous research [17], it yields 20% higher prediction accuracy. This improved prediction accuracy helps eliminate more unnecessary RDO operations.…”

Section: A Speedup Performancementioning

confidence: 64%

“…To solve the problems of data insufficiency and nonuniformity in training sample collection, we can perform data augmentation operations, such as rotation, noise addition, or partial cutting on data samples. However, data augmentation methods are not suitable to process intra-coded CU samples [17] in that the coder references neighboring block pixels for prediction. The coding mode of one CU would be different after flipping and rotation operations, i.e., one CU may comprise different labels.…”

Section: Figure 5: Blurred Regions Of Moving Objects In Videomentioning

confidence: 99%

“…It needs to preprocess the training samples after data augmentation to yield a database with uniformly distributed split modes. Previous works utilized CU textures and QP to train models to predict the 𝑚 𝑜 𝑝𝑡 for a CU [17]. However, VVC references neighboring block pixels to perform IP and calculates all RDCs to determine the optimal CU mode.…”

Section: B Coding Mode Handlingmentioning

confidence: 99%

See 2 more Smart Citations

Speed Up VVC Intra-Coding by Learned Models and Feature Statistics

Chen,

Chou,

Jiang

2023

IEEE Access

Self Cite

View full text Add to dashboard Cite

The newest video coding standard, Versatile Video Coding (VVC), adopts a QTMT, quadtree plus multi-type tree (MTT), block partition structure and improves the compression performance by about 30%∼50%, compared with the previous High-Efficiency Video Coding (HEVC) one, at the cost of higher time complexity. To make practical video communication applications feasible, we have to reduce the high time complexity resulting from an exhaustive rate-distortion optimization (RDO) search procedure. We proposed to predict Coding Unit (CU) split modes by a learned model whose input comprises neighboring line pixels and quantization parameters (QP). In addition, we set thresholds based on statistical image features and coding behaviors to eliminate unnecessary coding operations in critical coding control modules. Experiments showed that, compared with the default VVC intra-coding, the proposed method saves 46.73% of encoding time, with Bjøntegaard Delta Bit Rate (BDBR) increment of 1.16%. After retraining the learned model with a specified QP, the time reduction rate can achieve 51.79%, and the BDBR slightly increases to 2.07%. The proposed speedup coding scheme effectively reduced the VVC time complexity to a large extent.

show abstract

Section: A Speedup Performancementioning

confidence: 64%

Section: Figure 5: Blurred Regions Of Moving Objects In Videomentioning

confidence: 99%

Section: B Coding Mode Handlingmentioning

confidence: 99%

See 1 more Smart Citation

Speed Up VVC Intra-Coding by Learned Models and Feature Statistics

Chen,

Chou,

Jiang

2023

IEEE Access

Self Cite

View full text Add to dashboard Cite

show abstract

“…To more effectively assess the encoding efficiency of the algorithm, we combined the algorithm proposed in this paper with three different types of state-of-the-art algorithms, representing machine learning methods [35], deep learning methods [36] and traditional methods [37]. The configurations in Table 1 were used on VTM10.0 software, and the experimental results are shown in Table 2.…”

Section: Analysis Of Experimental Resultsmentioning

confidence: 99%

FSVM- and DAG-SVM-Based Fast CU-Partitioning Algorithm for VVC Intra-Coding

Wang

Zhang

2023

Symmetry

View full text Add to dashboard Cite

H.266/VVC introduces the QTMT partitioning structure, building upon the foundation laid by H.265/HEVC, which makes the partitioning more diverse and flexible but also brings huge coding complexity. To better address the problem, we propose a fast CU decision algorithm based on FSVMs and DAG-SVMs to reduce encoding time. The algorithm divides the CU-partitioning process into two stages and symmetrically extracts some of the same CU features. Firstly, CU is input into the trained FSVM model, extracting the standard deviation, directional complexity, and content difference complexity of the CUs, and it uses these features to make a judgment on whether to terminate the partitioning early. Then, the determination of the partition type of CU is regarded as a multi-classification problem, and a DAG-SVM classifier is used to classify it. The extracted features serve as input to the classifier, which predicts the partition type of the CU and thereby prevents unnecessary partitioning. The results of the experiment indicate that compared with the reference software VTM10.0 anchoring algorithm, the algorithm can save 49.38%~58.04% of coding time, and BDBR only increases by 0.76%~1.37%. The video quality and encoding performance are guaranteed while the encoding complexity is effectively reduced.

show abstract

“…The CU split type is then determined by comparing these probability values to a set threshold value. In the literature [35], to reduce coding time and bypass the RDO process, a novel algorithm that utilizes both CNN and random forest classifier (RFC) has been developed. This algorithm predicts the depth and split type of a 32 × 32 CU by combining the strengths of CNN and RFC in a fast CU split decision process.…”

Section: Background and Related Workmentioning

confidence: 99%

Fast CU Division Pattern Decision Based on the Combination of Spatio-Temporal Information

2023

View full text Add to dashboard Cite

In order to satisfy the growing need for high-quality video, VVC comes with more efficient coding performance. According to statistical analysis, the level of coding complexity in VVC is tenfold greater compared to that of HEVC, so it is our main goal to study that what methods can be employed to decrease the time complexity of VVC. CU split in intra-frame modes requires the split mode decision by RD loss calculation, and the process of coding makes it to calculate RD calculation for all possible mode combinations, which is an important area that brings complexity to video coding, so in order to achieve our goal. Initially, we introduce an optimal depth prediction algorithm for Coding Units (CUs) by leveraging temporal combination. This algorithm collects depth information of CUs to predict the coding depth of CU blocks. Additionally, we suggest a decision tree-based method for CU split mode decision. With this method, we can make a decision on the CU split mode within the obtained split depth, reducing the time complexity of coding. This decision is based on the predictions from the first algorithm. The results demonstrate that our algorithm achieves superior performance over state-of-the-art methods in terms of computational complexity and compression quality. Compared to the VVC reference software (VTM), our method saves an average of 53.92% in coding time and improves the BDBR by 1.74%. These findings suggest that our method is highly effective in improving both computational efficiency and compression quality.

show abstract

Speed Up H.266/QTMT Intra-Coding Based on Predictions of ResNet and Random Forest Classifier

Cited by 15 publications

References 9 publications

Speed Up VVC Intra-Coding by Learned Models and Feature Statistics

Speed Up VVC Intra-Coding by Learned Models and Feature Statistics

FSVM- and DAG-SVM-Based Fast CU-Partitioning Algorithm for VVC Intra-Coding

Fast CU Division Pattern Decision Based on the Combination of Spatio-Temporal Information

Contact Info

Product

Resources

About