Performance Evaluation of Objective Image Quality Metrics on Conventional and Learning-Based Compression Artifacts

Testolina, Michela; Upenik, Evgeniy; Ascenso, João; Pereira, Fernando; Ebrahimi, Touradj

doi:10.1109/qomex51781.2021.9465445

Cited by 14 publications

(22 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, the levels of performance attained by the PSNR family, which are commonly used in compression, were poor. Like other studies in the literature that analysed the perceptual quality of learning based image compression models [68], [69], our experiments also confirmed the value and perceptual relevance using models like MS-SSIM, VIF, and VMAF.…”

Section: E Subjective Study and Analysissupporting

confidence: 87%

“…Aside from measuring the pixel-wise PSNR for completeness, we mainly relied on perception-based quality models, including MS-SSIM [65], VIF [66] and VMAF [67], to quantify the distortion levels that were used for BD-rate calculation. We are aware of recent studies [68], [69] of quality evaluation on deep learning based image compression. It has been shown that in this context, these three perceptual quality models have the highest correlation against subjective scores, whereas absolute fidelity models like the PSNR correlate poorly with visual perception, producing significantly inferior quality predictions than perception-based quality predictors.…”

Section: A Evaluation Experimentsmentioning

confidence: 99%

See 1 more Smart Citation

Estimating the Resize Parameter in End-to-end Learned Image Compression

Chen¹,

Bampis²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

We describe a search-free resizing framework that can further improve the rate-distortion tradeoff of recent learned image compression models. Our approach is simple: compose a pair of differentiable downsampling/upsampling layers that sandwich a neural compression model. To determine resize factors for different inputs, we utilize another neural network jointly trained with the compression model, with the end goal of minimizing the rate-distortion objective. Our results suggest that "compression friendly" downsampled representations can be quickly determined during encoding by using an auxiliary network and differentiable image warping. By conducting extensive experimental tests on existing deep image compression models, we show results that our new resizing parameter estimation framework can provide Bjøntegaard-Delta rate (BD-rate) improvement of about 10% against leading perceptual quality engines. We also carried out a subjective quality study, the results of which show that our new approach yields favorable compressed images. To facilitate reproducible research in this direction, the implementation used in this paper is being made freely available online at: https://github.com/treammm/ResizeCompression.

show abstract

Section: E Subjective Study and Analysissupporting

confidence: 87%

Section: A Evaluation Experimentsmentioning

confidence: 99%

Estimating the Resize Parameter in End-to-end Learned Image Compression

Chen¹,

Bampis²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…It has been concluded that conventional PSNR and SSIM metrics are not appropriate to assess the quality of deep learning-based compressed images. A similar work using 8 reference images and focusing on traditional objective quality assessment metrics has also been presented in [52].…”

Section: Related Workmentioning

confidence: 99%

“…The distorted images, obtained with the standard JPEG2000 coding standard and some recent neural networks based compression algorithms, as well as their associated MOS are made publicly available. Moreover, compared to the previous subjective studies [52,55], the proposed database includes more reference images with recent NN compression algorithms resulting in a larger dataset. It is worth pointing out that such new subjective dataset presents a great interest to the research communities working both on the development of IQA algorithms as well as the design of deep learning based compression algorithms.…”

Section: Limitations and Contributionsmentioning

confidence: 99%

“…Based on this database, we achieve an extensive evaluation of image quality assessment metrics and analyse their correlation with the subjective quality scores. In this respect, and unlike recent studies [52,55] focusing on traditional IQA algorithms, we propose here to investigate new emerging IQA methods based on deep learning approaches. It is important to note here that the corresponding models have been trained on our database, which allows us to learn new models more adapted to the quality assessment of neural networks-based compressed images.…”

Section: Limitations and Contributionsmentioning

confidence: 99%

See 1 more Smart Citation

NNCD-IQA: A new neural networks based compressed database for image quality assessment

Khan

Dardouri²,

Kaaniche³

et al. 2022

Multimed Tools Appl

View full text Add to dashboard Cite

Objective and subjective quality assessment is still a challenging problem in various image processing tasks. For instance, in the context of image compression, most of the conducted studies have focused on image datasets encoded using standard algorithms such as JPEG and JPEG2000. In this paper, we propose to further investigate the quality assessment issue in the presence of neural networks-based compressed images. More precisely, a new database of compressed images has been firstly built using JPEG2000 standard as well as four recent neural networks based coding schemes. Then, subjective experiments are performed to obtain the mean opinion scores of the generated distorted images. Finally, an extensive evaluation and analysis of objective image quality assessment metrics is achieved. For instance, in addition to conventional and machine learning metrics, we have considered different deep learning based models, which have been trained on our database. The new subjective database with its associated mean opinion scores as well as the learned models are publicly available at https://github.com/zakopz/NNCD-IQA-Database. The obtained results show the interest of deep learning based metrics in the context of neural networks-based compressed images. Keywords Image compression• Neural networks • Quality assessment • Subjective scores • Learning based metrics Zohaib Amjad Khan and Tassnim Dardouri contributed equally.

show abstract

Towards a Multiscale Point Cloud Structural Similarity Metric

Lazzarotto,

Ebrahimi

2023

2023 IEEE 25th International Workshop on Multimedia Signal Processing (MMSP)

Self Cite

View full text Add to dashboard Cite

Point clouds are effective data structures for the representation of three-dimensional media and hence adopted in a wide range of practical applications. In many cases, the portrayed data is expected to be visualized by humans. After acquisition, point clouds may undergo different processing operations such as compression or denoising, potentially affecting their perceived quality. Although subjective experiments are still the most reliable form of assessing the intensity of degradation, they are expensive and time-consuming, pushing many systems to depend on objective metrics. Such algorithms are used to model the human visual system, and their performance is usually assessed through their correlation with subjective visual quality scores. In this paper, an objective quality metric capable of evaluating distortions between a reference and a distorted point cloud at multiple scales is presented. The proposed metric is based on the point cloud structural similarity metric (PointSSIM), which computes a score based on the difference between statistical estimators obtained on the distribution of the luminance attribute over local neighborhoods. A collection of PointSSIM scores is produced for multiple scales obtained through the voxelization of both models at different bit depth precisions. These scores are then pooled through a weighted sum, with the importance of each scale being defined through logistic fitting to subjective mean opinion scores, producing one MS-PointSSIM score. Three datasets were employed for fitting and performance assessment, demonstrating a clear advantage of the proposed metric when compared to the single-scale baseline. Moreover, the presented MS-PointSSIM is shown to be the best predictor according to the average Pearson correlation coefficient across the three datasets when compared to state-of-the-art metrics.

show abstract

Performance Evaluation of Objective Image Quality Metrics on Conventional and Learning-Based Compression Artifacts

Cited by 14 publications

References 16 publications

Estimating the Resize Parameter in End-to-end Learned Image Compression

Estimating the Resize Parameter in End-to-end Learned Image Compression

NNCD-IQA: A new neural networks based compressed database for image quality assessment

Towards a Multiscale Point Cloud Structural Similarity Metric

Contact Info

Product

Resources

About