Video Compression Using Generalized Binary Partitioning, Trellis Coded Quantization, Perceptually Optimized Encoding, and Advanced Prediction and Transform Coding

Abstract

In this paper, we describe a video coding design that enables a higher coding efficiency than the HEVC standard. The proposed video codec follows the design of block-based hybrid video coding, but includes a number of advanced coding tools. A part of the incorporated advanced concepts was developed by the Joint Video Exploration Team, while others are newly proposed. The key aspects of these newly proposed tools are the following. A video frame is subdivided into rectangles of variable size using a binary partitioning with variable split ratios. Three new approaches for generating spatial intra prediction signals are supported: A line-wise application of conventional intra prediction modes, coupled with a mode-dependent processing order, a region-based template matching prediction method and intra prediction modes based on neural networks. For motion-compensated prediction, a multi-hypothesis mode with more than two motion hypotheses can be used. In transform coding, mode dependent combinations of primary and secondary transforms are applied. Moreover, scalar quantization is replaced by trellis-coded quantization and the entropy coding of the quantized transform coefficients is improved. The intra and inter prediction signals can be filtered using an edge-preserving diffusion filter or a non-linear DCT-based thresholding operation.

Sequence-Level Reference Frames in Video Coding

“…However, the reported gains are comparable or superior to those of tools that have been considered for inclusion in the VVC standard [34]. For example, Pfaff et al [57] present an analysis showing that 8 out of the 14 new coding tools (Table III of their paper [57]) achieve BD-rates of -0.7% to -2.8%, but they cumulatively contribute to BDrate of -21% over the HM reference software. Such scaleup of performance is observed in our proposal when scene repetition occurs more frequently in the input video: Table VI shows that sequence-level references provide for up to -33% in BD-rate for PSNR or VMAF in comparison to VVC.…”

Intra Prediction and Mode Coding in VVC

“…1) Layout: Given a W ×H block of width W and height H, the dimensions of the corresponding subpartitions are W × H K if the split is horizontal and W K × H if it is vertical. In earlier versions of the algorithm [28][29] [30], K was equal to H (horizontal split) or W (vertical split), so that the resulting subpartitions would exclusively become onedimensional blocks or lines. Although such an approach provides a benefit in terms of prediction gain [31], it does not exploit the potential 2-D transform gain [31] and it can increase the rate overhead excessively in cases where the total number of subpartitions is very large.…”

“…2) Processing: The subpartitions are processed one-by-one from the top to the bottom if the split is horizontal and from the left to the right if it is vertical. Despite the fact that previous versions of the algorithm [28][29] [30] incorporated reversed variants of these arrangements (from the bottom to the top and from the right to the left) for certain combinations of split type and angular intra mode, they were finally not included in the standard to avoid an increase in the implementation complexity.…”

“…6) MIP as a Video-Coding Tool Designed by Data-Driven Methods: The final design of the MIP-modes in VVC can be regarded as a low-complexity variant of neural networkbased intra prediction modes that were initially presented in [30], [38], [39]. While key aspects of the data-driven training algorithm of [30] were also employed to determine the parameters of the matrices A k , the design finally adopted to VVC represents a different operational point in the gaincomplexity tradeoff than the initial variant of [30]. The central steps for the complexity reduction are the downsampling of the input and the upsampling of the output of the prediction.…”

Block Partitioning Structure in the VVC Standard

“…In April 2018, VCEG and MPEG received 22, 12, and 12 responses in the Standard Dynamic Range (SDR), High Dynamic Range (HDR), and 360 • categories, respectively [8]. Many proposals contributed as responses to the CfP, e.g., [9]- [14], showed significant coding gains over the HM and even over the JEM. The Joint Video Exploration Team was renamed as the Joint Video Experts Team (JVET), which then officially started the standardization process of Versatile Video Coding (VVC) | ITU-T Rec.…”