Learning True Rate-Distortion-Optimization for End-To-End Image Compression

Abstract

Even though rate-distortion optimization is a crucial part of traditional image and video compression, not many approaches exist which transfer this concept to end-to-end-trained image compression. Most frameworks contain static compression and decompression models which are fixed after training, so efficient rate-distortion optimization is not possible. In a previous work, we proposed RDONet, which enables an RDO approach comparable to adaptive block partitioning in HEVC. In this paper, we enhance the training by introducing low-complexity estimations of the RDO result into the training. Additionally, we propose fast and very fast RDO inference modes. With our novel training method, we achieve average rate savings of 19.6% in MS-SSIM over the previous RDONet model, which equals rate savings of 27.3% over a comparable conventional deep image coder.

Spatial Rate Allocation for Learning-based Video Coding

“…RDOnet deploys masking layers to zero-out certain coefficients. By training models with such layers, unimportant regions of the image are identified during inference and do not have their information transmitted [11]- [13]. In [14], an ROI-based multi-rate codec is proposed that can dynamically control local and global rate allocation at the frame-level.…”

GOP-Based Latent Refinement for Learned Video Coding

“…For instance, RDOnet deploys masking layers to zero-out certain coefficients. By training models with such layers, unimportant regions of the image are identified during inference and do not have their information transmitted [7]- [9]. The drawback of this approach is the lack of inference-time signal adaptation.…”