Perceptually Optimizing Deep Image Compression

Chen, Li-Heng; Bampis, Christos G.; Li, Zhi; Norkin, Andrey; Bovik, Alan C.

doi:10.48550/arxiv.2007.02711

Cited by 3 publications

(4 citation statements)

References 37 publications

(41 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…in which the frequency index k is dropped for simplicity. As can be observed, the statistics of A in (12) are the same as A (sub) in (34). Thus, without any loss of generality we find the distribution of A (sub) .…”

Section: Appendix a Proof Of Theoremmentioning

confidence: 51%

“…Another difference is that a good performance metric for channel estimation is the Euclidean distance or SNR, due to Gaussian noise and the dispassionate nature of symbol demodulation in the presence of such noise. On the other hand, image quality is perceptual and qualitative, and Euclidean distance and SNR are known to be poor measures of image quality [33], [34]. Indeed, a major feature of a GAN is that it can produce an image that is far from the target image under a quantitative measure like Euclidean distance, but very close in a perceptual sense.…”

Section: Channel Estimation and Image Reconstruction Differences For ...mentioning

confidence: 99%

See 1 more Smart Citation

Wideband Channel Estimation with A Generative Adversarial Network

Balevi¹,

Andrews²

2020

Preprint

View full text Add to dashboard Cite

Communication at high carrier frequencies such as millimeter wave (mmWave) and terahertz (THz) requires channel estimation for very large bandwidths at low SNR. Hence, allocating an orthogonal pilot tone for each coherence bandwidth leads to excessive number of pilots. We leverage generative adversarial networks (GANs) to accurately estimate frequency selective channels with few pilots at low SNR. The proposed estimator first learns to produce channel samples from the true but unknown channel distribution via training the generative network, and then uses this trained network as a prior to estimate the current channel by optimizing the network's input vector in light of the current received signal. Our results show that at an SNR of −5 dB, even if a transceiver with one-bit phase shifters is employed, our design achieves the same channel estimation error as an LS estimator with SNR = 20 dB or the LMMSE estimator at 2.5 dB, both with fully digital architectures. Additionally, the GAN-based estimator reduces the required number of pilots by about 70% without significantly increasing the estimation error and required SNR. We also show that the generative network does not appear to require retraining even if the number of clusters and rays change considerably.

show abstract

Section: Appendix a Proof Of Theoremmentioning

confidence: 51%

Section: Channel Estimation and Image Reconstruction Differences For ...mentioning

confidence: 99%

Wideband Channel Estimation with A Generative Adversarial Network

Balevi¹,

Andrews²

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Recent developments, such as the JPEG-AI competition, 1 focus on image compression methods that are learning-based and suitable for higher-resolution images. Those learningbased methods can be implemented using DNNs [10,62,116] or use hybrid approaches that rely on traditional methods combined with neural networks for image enhancement [56]. An example of such a hybrid variant is proposed by Lee et al [56].…”

Section: High Resolution Image Quality Assessmentmentioning

confidence: 99%

Quality assessment of higher resolution images and videos with remote testing

2023

View full text Add to dashboard Cite

In many research fields, human-annotated data plays an important role as it is used to accomplish a multitude of tasks. One such example is in the field of multimedia quality assessment where subjective annotations can be used to train or evaluate quality prediction models. Lab-based tests could be one approach to get such quality annotations. They are usually performed in well-defined and controlled environments to ensure high reliability. However, this high reliability comes at a cost of higher time consumption and costs incurred. To mitigate this, crowd or online tests could be used. Usually, online tests cover a wider range of end devices, environmental conditions, or participants, which may have an impact on the ratings. To verify whether such online tests can be used for visual quality assessment, we designed three online tests. These online tests are based on previously conducted lab tests as this enables comparison of the results of both test paradigms. Our focus is on the quality assessment of high-resolution images and videos. The online tests use AVrate Voyager, which is a publicly accessible framework for online tests. To transform the lab tests into online tests, dedicated adaptations in the test methodologies are required. The considered modifications are, for example, a patch-based or centre cropping of the images and videos, or a randomly sub-sampling of the to-be-rated stimuli. Based on the analysis of the test results in terms of correlation and SOS analysis it is shown that online tests can be used as a reliable replacement for lab tests albeit with some limitations. These limitations relate to, e.g., lack of appropriate display devices, limitation of web technologies, and modern browsers considering support for different video codecs and formats.

show abstract

“…Moreover, it is well known that 1 and 2 losses do not correlate well with subjective video quality [67][68][69], and the combined loss functions employed in these GANbased training strategies use artificially configured combining weights, which have never been fully evaluated in terms of their correlation with subjective video quality. These issues inevitably lead to sub-optimal training performance when the networks are utilised for compression application.…”

Section: Training Strategymentioning

confidence: 99%

CVEGAN: A Perceptually-inspired GAN for Compressed Video Enhancement

Ma¹,

Zhang²,

Bull³

2020

Preprint

View full text Add to dashboard Cite

We propose a new Generative Adversarial Network for Compressed Video quality Enhancement (CVEGAN). The CVEGAN generator benefits from the use of a novel Mul 2 Res block (with multiple levels of residual learning branches), an enhanced residual non-local block (ERNB) and an enhanced convolutional block attention module (ECBAM). The ERNB has also been employed in the discriminator to improve the representational capability. The training strategy has also been re-designed specifically for video compression applications, to employ a relativistic sphere GAN (ReSphereGAN) training methodology together with new perceptual loss functions. The proposed network has been fully evaluated in the context of two typical video compression enhancement tools: post-processing (PP) and spatial resolution adaptation (SRA). CVEGAN has been fully integrated into the MPEG HEVC video coding test model (HM16.20) and experimental results demonstrate significant coding gains (up to 28% for PP and 38% for SRA compared to the anchor) over existing state-of-the-art architectures for both coding tools across multiple datasets.

show abstract

Perceptually Optimizing Deep Image Compression

Cited by 3 publications

References 37 publications

Wideband Channel Estimation with A Generative Adversarial Network

Wideband Channel Estimation with A Generative Adversarial Network

Quality assessment of higher resolution images and videos with remote testing

CVEGAN: A Perceptually-inspired GAN for Compressed Video Enhancement

Contact Info

Product

Resources

About