High-Fidelity Generative Image Compression

Abstract

We extensively study how to combine Generative Adversarial Networks and learned compression to obtain a state-of-the-art generative lossy compression system. In particular, we investigate normalization layers, generator and discriminator architectures, training strategies, as well as perceptual losses. In contrast to previous work, i) we obtain visually pleasing reconstructions that are perceptually similar to the input, ii) we operate in a broad range of bitrates, and iii) our approach can be applied to high-resolution images. We bridge the gap between rate-distortionperception theory and practice by evaluating our approach both quantitatively with various perceptual metrics and a user study. The study shows that our method is preferred to previous approaches even if they use more than 2× the bitrate.

SoundStream: An End-to-End Neural Audio Codec

“…Instead, decreasing the capacity of the decoder has a more significant impact on quality (ViSQOL drops from 3.96 to 3.84). This is aligned with recent findings in the field of neural image compression [61], which also adopt a lighter encoder and a heavier decoder.…”

SoundStream: An End-to-End Neural Audio Codec

“…Instead, decreasing the capacity of the decoder has a more significant impact on quality (ViSQOL drops from 3.96 to 3.84). This is aligned with recent findings in the field of neural image compression [61], which also adopt a lighter encoder and a heavier decoder.…”

End-to-end optimized image compression for machines, a study

“…ANN-based codecs have already been proven to reach performances close to conventional methods, and to outperform them when trained with perceptual quality metrics [6,7,9,10]. This paper highlights another clear advantage of ANN-based codecs: optimized compression for machines.…”

Task-Adaptive Decoder Specialization for Unified Image Coding Toward Human Reconstruction and Machine Classification

“…6 (left) and Table 2, Ours(Full MSE) obtains 29.78 dB at 0.15 bpp on Kodak, exceeding MPA [36], MRIC [2], CRDR [17], and TinyLIC [27] by 0.98 dB, 0.64 dB, 0.99 dB, and 0.41 dB, respectively. The BD metrics further show that Ours(Full MSE) achieves a 54.23% BD-Rate reduction and a 2.81 dB BD-PSNR gain relative to HiFiC [28], which is higher than the corresponding gains of MPA and other compared methods. It should be noted that generative or controllable coding methods such as MRIC, CRDR, and HiFiC are @0.15bpp @0.25bpp @0.35bpp @0.50bpp @0.70bpp BD-Rate ↓ BD-Acc ↑ VTM17.…”

SoundStream: An End-to-End Neural Audio Codec

“…Instead, decreasing the capacity of the decoder has a more significant impact on quality (ViSQOL drops from 3.96 to 3.84). This is aligned with recent findings in the field of neural image compression [61], which also adopt a lighter encoder and a heavier decoder.…”

End-to-end optimized image compression for machines, a study

“…ANN-based codecs have already been proven to reach performances close to conventional methods, and to outperform them when trained with perceptual quality metrics [6,7,9,10]. This paper highlights another clear advantage of ANN-based codecs: optimized compression for machines.…”

Task-Adaptive Decoder Specialization for Unified Image Coding Toward Human Reconstruction and Machine Classification

“…6 (left) and Table 2, Ours(Full MSE) obtains 29.78 dB at 0.15 bpp on Kodak, exceeding MPA [36], MRIC [2], CRDR [17], and TinyLIC [27] by 0.98 dB, 0.64 dB, 0.99 dB, and 0.41 dB, respectively. The BD metrics further show that Ours(Full MSE) achieves a 54.23% BD-Rate reduction and a 2.81 dB BD-PSNR gain relative to HiFiC [28], which is higher than the corresponding gains of MPA and other compared methods. It should be noted that generative or controllable coding methods such as MRIC, CRDR, and HiFiC are @0.15bpp @0.25bpp @0.35bpp @0.50bpp @0.70bpp BD-Rate ↓ BD-Acc ↑ VTM17.…”

SoundStream: An End-to-End Neural Audio Codec

“…Instead, decreasing the capacity of the decoder has a more significant impact on quality (ViSQOL drops from 3.96 to 3.84). This is aligned with recent findings in the field of neural image compression [61], which also adopt a lighter encoder and a heavier decoder.…”

End-to-end optimized image compression for machines, a study

“…ANN-based codecs have already been proven to reach performances close to conventional methods, and to outperform them when trained with perceptual quality metrics [6,7,9,10]. This paper highlights another clear advantage of ANN-based codecs: optimized compression for machines.…”

Task-Adaptive Decoder Specialization for Unified Image Coding Toward Human Reconstruction and Machine Classification

“…6 (left) and Table 2, Ours(Full MSE) obtains 29.78 dB at 0.15 bpp on Kodak, exceeding MPA [36], MRIC [2], CRDR [17], and TinyLIC [27] by 0.98 dB, 0.64 dB, 0.99 dB, and 0.41 dB, respectively. The BD metrics further show that Ours(Full MSE) achieves a 54.23% BD-Rate reduction and a 2.81 dB BD-PSNR gain relative to HiFiC [28], which is higher than the corresponding gains of MPA and other compared methods. It should be noted that generative or controllable coding methods such as MRIC, CRDR, and HiFiC are @0.15bpp @0.25bpp @0.35bpp @0.50bpp @0.70bpp BD-Rate ↓ BD-Acc ↑ VTM17.…”