From Patches to Pictures (PaQ-2-PiQ): Mapping the Perceptual Space of Picture Quality

Ying, Zhenqiang; Niu, Haoran; Gupta, P. K. Sen; Mahajan, Dhruv; Ghadiyaram, Deepti; Bovik, Alan C.

doi:10.48550/arxiv.1912.10088

Cited by 4 publications

(3 citation statements)

References 64 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…MLSP-FF [8] also uses heavier Inception-ResNet-V2 [33] for feature extraction. Some methods [40,41] use the features extractor pre-trained with IQA datasets [13,39]. PVQ [40] also extracts features pretrained on action recognition dataset [16] for better perception on inter-frame distortion.…”

Section: Related Workmentioning

confidence: 99%

FAST-VQA: Efficient End-to-end Video Quality Assessment with Fragment Sampling

Wu¹,

Chen²,

Hou³

et al. 2022

Preprint

View full text Add to dashboard Cite

Current deep video quality assessment (VQA) methods are usually with high computational costs when evaluating high-resolution videos. This cost hinders them from learning better video-quality-related representations via end-to-end training. Existing approaches usually consider naive sampling to reduce the computational cost, such as resizing and cropping. However, they obviously corrupt quality-related information in videos and are thus not optimal to learn good representations for VQA. Therefore, there is an eager need to design a new quality-retained sampling scheme for VQA. In this paper, we propose Grid Mini-patch Sampling (GMS), which allows consideration of local quality by sampling patches at their raw resolution and covers global quality with contextual relations via mini-patches sampled in uniform grids. These mini-patches are spliced and aligned temporally, named as fragments. We further build the Fragment Attention Network (FANet) specially designed to accommodate fragments as inputs. Consisting of fragments and FANet, the proposed FrAgment Sample Transformer for VQA (FAST-VQA) enables efficient end-to-end deep VQA and learns effective video-qualityrelated representations. It improves state-of-the-art accuracy by around 10% while reducing 99.5% FLOPs on 1080P high-resolution videos. The newly learned video-quality-related representations can also be transferred into smaller VQA datasets and boost the performance on these scenarios. Extensive experiments show that FAST-VQA has good performance on inputs of various resolutions while retaining high efficiency. We publish our code at https://github.com/timothyhtimothy/FAST-VQA.

show abstract

Section: Related Workmentioning

confidence: 99%

FAST-VQA: Efficient End-to-end Video Quality Assessment with Fragment Sampling

Wu¹,

Chen²,

Hou³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Early models were often distortion-specific, targeting only one or more specific distortions. Later, more general models have involved training quality prediction models to learn mappings from features to subjective judgments, e.g., BRISQUE [27], V-BLIINDS [28] and BIQME [29], which make use of natural scene statistics (NSS) models; BPRI [30], [31], which utilizes a pseudo reference image, TLVQM [32], which deploys a simplified motion estimator, and more recent deep learning models, like NIMA [33], PaQ-2-PiQ [34], PQR [35], DLIQA [36], and more [37]. There are also completely blind (unsupervised) models, like NIQE [38] and IL-NIQE [39].…”

Section: Related Workmentioning

confidence: 99%

Predicting the Quality of Compressed Videos with Pre-Existing Distortions

Yu,

Birkbeck,

Wang

et al. 2020

Preprint

View full text Add to dashboard Cite

Over the past decade, the online video industry has greatly expanded the volume of visual data that is streamed and shared over the Internet. Moreover, because of the increasing ease of video capture, many millions of consumers create and upload large volumes of User-Generated-Content (UGC) videos. Unlike streaming television or cinematic content produced by professional videographers and cinemagraphers, UGC videos are most commonly captured by naive users having limited skills and imperfect technique, and often are afflicted by highly diverse and mixed in-capture distortions. These UGC videos are then often uploaded for sharing onto cloud servers, where they further compressed for storage and transmission. Our paper tackles the highly practical problem of predicting the quality of compressed videos (perhaps during the process of compression, to help guide it), with only (possibly severely) distorted UGC videos as references. To address this problem, we have developed a novel Video Quality Assessment (VQA) framework that we call 1stepVQA (to distinguish it from two-step methods that we discuss). 1stepVQA overcomes limitations of Full-Reference, Reduced-Reference and No-Reference VQA models by exploiting the statistical regularities of both natural videos and distorted videos. We show that 1stepVQA is able to more accurately predict the quality of compressed videos, given imperfect reference videos. We also describe a new dedicated video database which includes (typically distorted) UGC reference videos, and a large number of compressed versions of them. We show that the 1stepVQA model outperforms other VQA models in this scenario. We are providing the dedicated new database free of charge at https://live.ece.utexas.edu/research/onestep/index.html

show abstract

“…With an interest in its possibilities, Deepti Ghadiyaram and I published one of the earliest papers, if not the first, on picture quality prediction using deep (belief) nets [5]. Unfortunately, when trained and tested on the new LIVE Challenge picture quality database, which was the largest database then available [6] (but see [7]), the deep models could not attain the performance of simple natural scene-based models like BRISQUE [8] and FRIQUEE [9]. Rigorous tests on other researchers' deep models produced similar results [10].…”

Section: Denialmentioning

confidence: 99%

Weeping and Gnashing of Teeth: Teaching Deep Learning in Image and Video Processing Classes

Bovik

2020

2020 IEEE Southwest Symposium on Image Analysis and Interpretation (SSIAI)

View full text Add to dashboard Cite

In this rather informal paper and talk I will discuss my own experiences, feelings and evolution as an Image Processing and Digital Video educator trying to navigate the Deep Learning revolution. I will discuss my own ups and downs of trying to deal with extremely rapid technological changes, and how I have reacted to, and dealt with consequent dramatic changes in the relevance of the topics I've taught for three decades. I have arranged the discussion in terms of the stages, over time, of my progression dealing with these sea changes.

show abstract

From Patches to Pictures (PaQ-2-PiQ): Mapping the Perceptual Space of Picture Quality

Cited by 4 publications

References 64 publications

FAST-VQA: Efficient End-to-end Video Quality Assessment with Fragment Sampling

FAST-VQA: Efficient End-to-end Video Quality Assessment with Fragment Sampling

Predicting the Quality of Compressed Videos with Pre-Existing Distortions

Weeping and Gnashing of Teeth: Teaching Deep Learning in Image and Video Processing Classes

Contact Info

Product

Resources

About