Better Compression With Deep Pre-Editing

Abstract

Could we compress images via standard codecs while avoiding visible artifacts? The answer is obvious -this is doable as long as the bit budget is generous enough. What if the allocated bit-rate for compression is insufficient? Then unfortunately, artifacts are a fact of life. Many attempts were made over the years to fight this phenomenon, with various degrees of success. In this work we aim to break the unholy connection between bit-rate and image quality, and propose a way to circumvent compression artifacts by pre-editing the incoming image and modifying its content to fit the given bits. We design this editing operation as a learned convolutional neural network, and formulate an optimization problem for its training. Our loss takes into account a proximity between the original image and the edited one, a bit-budget penalty over the proposed image, and a no-reference image quality measure for forcing the outcome to be visually pleasing. The proposed approach is demonstrated on the popular JPEG compression, showing savings in bits and/or improvements in visual quality, obtained with intricate editing effects.

Neural JPEG: End-to-End Image Compression Leveraging a Standard JPEG Encoder-Decoder

“…A more straightforward compression approach considered redundancy at the decoder side of the system and attempted to decompress by designing an iterative hybrid recurrent decoder [6,7,8]. Similarly, a standard encoder can be replaced with another DNN to enhance the model's internal neural representations and decode information while only using a standard decoder both in the pixel [24] and frequency domains [10].…”

“…Editing with Sparse RNNs: Inspired by [26] and [10], we design an approach that uses a neural models to either pre-edit (or iteratively process) an image I before the quantization step of JPEG or post-edit the inverse DCT coefficients before converting back to the reconstructed image Î. Specifically, for pre-editing, our neural encoder E Θ (I) proceeds according to the following steps to produce a set of "edit" weights:…”

Learning to Improve Image Compression without Changing the Standard Decoder

“…To overcome this shortcoming, this paper proposes improving rate-distortion performance without modifying the standard JPEG decoder. As far as we know, Talebi et al [29] is the only work on pre-editing before JPEG compression, which trains a DNN in the pixel domain before the JPEG encoder to pre-edit input images. Different from [29], we propose learning to improve the JPEG encoder in the frequency domain, i.e., learning an attention map to apply spatial weighting to the DCT coefficients and learning the quantization tables to optimize rate-distortion performance.…”

“…We propose a novel approach to preediting the image before quantization to improve the compression quality. [29] has shown that an image smoothing network before compressing the image improves the compression performance. We also employ a smoothing mechanism that acts on the DCT coefficients directly.…”

“…Because of the previously mentioned differentiability issue, we cannot directly add the entropy as a loss term to be minimized. Instead, [29] uses the differentiable entropy estimator proposed in [6] that relies on a continuous density estimator explained in the appendix of [7]. Using this estimator to optimize our learned JPEG architecture did not result in improved performance over the standard JPEG.…”