Machine Learning Based Fast QTMTT Partitioning Strategy for VVenC Encoder in Intra Coding

Abstract

The newest video compression standard, Versatile Video Coding (VVC), was finalized in July 2020 by the Joint Video Experts Team (JVET). Its main goal is to reduce the bitrate by 50% over its predecessor video coding standard, the High Efficiency Video Coding (HEVC). Due to the new advanced tools and features included in VVC, it actually provides high coding performances—for instance, the Quad Tree with nested Multi-Type Tree (QTMTT) involved in the partitioning block. Furthermore, VVC introduces various techniques that allow for superior performance compared to HEVC, but with an increase in the computational complexity. To tackle this complexity, a fast Coding Unit partition algorithm based on machine learning for the intra configuration in VVC is proposed in this work. The proposed algorithm is formed by five binary Light Gradient Boosting Machine (LightGBM) classifiers, which can directly predict the most probable split mode for each coding unit without passing through the exhaustive process known as Rate Distortion Optimization (RDO). These LightGBM classifiers were offline trained on a large dataset; then, they were embedded on the optimized implementation of VVC known as VVenC. The results of our experiment show that our proposed approach has good trade-offs in terms of time-saving and coding efficiency. Depending on the preset chosen, our approach achieves an average time savings of 30.21% to 82.46% compared to the VVenC encoder anchor, and a Bjøntegaard Delta Bitrate (BDBR) increase of 0.67% to 3.01%, respectively.

A Fast Algorithm for Intra-Frame Versatile Video Coding Based on Edge Features

“…Since more partition modes are skipped under this condition, more time saving is achieved with a slight increase in BDBR loss. The method proposed in [15] saves 31.44% encoding time with a BDBR loss of 0.59%. Our method reduces the coding time by 32.52%, while maintaining better coding performance.…”

“…In [14], a fast algorithm based on machine learning is proposed, which uses texture complexity to determine the division direction and a lightweight neural network to determine the division mode. In [15], the authors present a low-complexity method, which is formed of five binary Light Gradient Boosting Machine (LightGBM) classifiers. In [16], a fast CU partitioning decision algorithm is presented based on texture complexity and convolutional neural networks (CNNs).…”

“…In [25], a latitude-based preprocessing is introduced to enable early termination of the coding unit (CU) partition in the polar region. In [15], the authors apply the factor of average background luminance for just-noticeable distortion to identify the visually distinguishable (VD) pixels within a coding unit (CU).…”

Light-Weight CNN-Based VVC Inter Partitioning Acceleration

“…For example, VVenc [41] is a fast implementation of VVC. In the All Intra configuration, VTM10.0 is reported to be 27 times more complex compared to VVenc with fast preset, as mentioned in [42]. The overall complexity of the CNNbased method presented in [17] accounts for only 2.34% of the encoding time of the VTM10 encoder.…”

Coding Tree Unit Partition Prediction Algorithm Utilizing Residual AttentionBased Long Short-Term Memory for Video Intra Coding

“…This increasing introduced of video information leads to issues on storage and transmission [2]. For transmitting a huge quantity of video information effectively at a better bit rate, the standards of video transmission are introduced like multiple function video coding standard (VVC/H.266), advanced video coding (AVC/H.264), and high-efficiency video coding (HEVC/H.265) [3]. The video coding standards mentioned in the text employ flexible block coding techniques and incorporate effective coding tools, thereby maximizing complexity of different video coding standards [4].…”