Fidelity-Controllable Extreme Image Compression with Generative Adversarial Networks

Iwai, Shoma; Miyazaki, Tomo; Sugaya, Yoshihiro; Omachi, Shinichiro

doi:10.1109/icpr48806.2021.9412185

Cited by 28 publications

(14 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The work ) lays a first theoretical foundation for understanding the distortion-perception (D-P) tradeoff, revealing that distortion and perception quality are at odds with each other. It accords well with empirical results that improving perceptual quality by adversarial learning and/or deep features based perceptual loss would lead to an increase of the distortion (Galteri et al, 2017;Santurkar et al, 2018;Agustsson et al, 2019;Iwai et al, 2020;Mentzer et al, 2020;Ohayon et al, 2021;Prakash et al, 2021).…”

Section: Introductionsupporting

confidence: 87%

See 1 more Smart Citation

Optimally Controllable Perceptual Lossy Compression

Yan¹,

Wen²,

Liu³

2022

Preprint

View full text Add to dashboard Cite

Recent studies in lossy compression show that distortion and perceptual quality are at odds with each other, which put forward the tradeoff between distortion and perception (D-P). Intuitively, to attain different perceptual quality, different decoders have to be trained. In this paper, we present a nontrivial finding that only two decoders are sufficient for optimally achieving arbitrary (an infinite number of different) D-P tradeoff. We prove that arbitrary points of the D-P tradeoff bound can be achieved by a simple linear interpolation between the outputs of a minimum MSE decoder and a specifically constructed perfect perceptual decoder. Meanwhile, the perceptual quality (in terms of the squared Wasserstein-2 distance metric) can be quantitatively controlled by the interpolation factor. Furthermore, to construct a perfect perceptual decoder, we propose two theoretically optimal training frameworks. The new frameworks are different from the distortionplus-adversarial loss based heuristic framework widely used in existing methods, which are not only theoretically optimal but also can yield state-of-the-art performance in practical perceptual decoding. Finally, we validate our theoretical finding and demonstrate the superiority of our frameworks via experiments.

show abstract

Section: Introductionsupporting

confidence: 87%

“…Even though it only requires to train a single encoder, different decoders need to be trained for different D-P tradeoff. (Iwai et al, 2020) considers the tradeoff between distortion and fidelity by interpolating the parameters or the outputs of two decoders. However, the method is heuristic and not optimal.…”

Section: Distortion-perception Tradeoffmentioning

confidence: 99%

Optimally Controllable Perceptual Lossy Compression

Yan¹,

Wen²,

Liu³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Other alternatives of BCE adversarial loss include leastsquare form [22] and relativistic form [19], which are also adopted by recent perceptual LIC approaches [12,18].…”

Section: Perceptual Optimization With Snganmentioning

confidence: 99%

PO-ELIC: Perception-Oriented Efficient Learned Image Coding

He¹,

Yang²,

Hou³

et al. 2022

Preprint

View full text Add to dashboard Cite

In the past years, learned image compression (LIC) has achieved remarkable performance. The recent LIC methods outperform VVC in both PSNR and MS-SSIM. However, the low bit-rate reconstructions of LIC suffer from artifacts such as blurring, color drifting and texture missing. Moreover, those varied artifacts make image quality metrics correlate badly with human perceptual quality. In this paper, we propose PO-ELIC, i.e., Perception-Oriented Efficient Learned Image Coding. To be specific, we adapt ELIC, one of the state-of-the-art LIC models, with adversarial training techniques. We apply a mixture of losses including hinge-form adversarial loss, Charbonnier loss, and style loss, to finetune the model towards better perceptual quality. Experimental results demonstrate that our method achieves comparable perceptual quality with HiFiC with much lower bitrate.

show abstract

“…The second is to incorporate an adversarial loss by using generative adversarial networks (GAN) (Goodfellow et al, 2014). Noteworthily, using an adversarial loss, the second method has shown remarkable effectiveness in achieving high perceptual quality (Rippel & Bourdev, 2017;Agustsson et al, 2019;Ledig et al, 2017;Wang et al, 2018;Wu et al, 2020;Iwai et al, 2020;Mentzer et al, 2020).…”

Section: Introductionmentioning

confidence: 99%

“…Our analysis also shows that an encoder optimized in terms of MSE is also an optimal encoder under perfect perception constraint. This result implies that the commonly used adversarial loss in state-of-the-art works (Blau & Michaeli, 2019;Agustsson et al, 2019;Iwai et al, 2020) is in fact unnecessary for optimizing the encoder.…”

Section: Introductionmentioning

confidence: 99%

On Perceptual Lossy Compression: The Cost of Perceptual Reconstruction and An Optimal Training Framework

Yan,

Wen,

Ying

et al. 2021

Preprint

View full text Add to dashboard Cite

Lossy compression algorithms are typically designed to achieve the lowest possible distortion at a given bit rate. However, recent studies show that pursuing high perceptual quality would lead to increase of the lowest achievable distortion (e.g., MSE). This paper provides nontrivial results theoretically revealing that, 1) the cost of achieving perfect perception quality is exactly a doubling of the lowest achievable MSE distortion, 2) an optimal encoder for the "classic" rate-distortion problem is also optimal for the perceptual compression problem, 3) distortion loss is unnecessary for training a perceptual decoder. Further, we propose a novel training framework to achieve the lowest MSE distortion under perfect perception constraint at a given bit rate. This framework uses a GAN with discriminator conditioned on an MSE-optimized encoder, which is superior over the traditional framework using distortion plus adversarial loss. Experiments are provided to verify the theoretical finding and demonstrate the superiority of the proposed training framework.

show abstract

Fidelity-Controllable Extreme Image Compression with Generative Adversarial Networks

Cited by 28 publications

References 10 publications

Optimally Controllable Perceptual Lossy Compression

Optimally Controllable Perceptual Lossy Compression

PO-ELIC: Perception-Oriented Efficient Learned Image Coding

On Perceptual Lossy Compression: The Cost of Perceptual Reconstruction and An Optimal Training Framework

Contact Info

Product

Resources

About