Empirical evaluation of no-reference VQA methods on a natural video quality database

Men, Hui; Saupe, Dietmar

doi:10.1109/qomex.2017.7965644

Cited by 15 publications

(19 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These deficiencies of current video databases would be difficult to assess quantitatively. However, an indirect confirmation is given by the fact that the performance of two established objective VQA algorithms on our KoNViD-1k database was significantly worse than on the traditional databases that were used for their development, even when the techniques were trained on our natural video dataset [23].…”

Section: Related Workmentioning

confidence: 90%

The Konstanz natural video database (KoNViD-1k)

Hosu

Hahn

Jenadeleh

et al. 2017

2017 Ninth International Conference on Quality of Multimedia Experience (QoMEX)

Self Cite

240

245

View full text Add to dashboard Cite

Subjective video quality assessment (VQA) strongly depends on semantics, context, and the types of visual distortions. Currently, all existing VQA databases include only a small number of video sequences with artificial distortions. The development and evaluation of objective quality assessment methods would benefit from having larger datasets of real-world video sequences with corresponding subjective mean opinion scores (MOS), in particular for deep learning purposes. In addition, the training and validation of any VQA method intended to be 'general purpose' requires a large dataset of video sequences that are representative of the whole spectrum of available video content and all types of distortions. We report our work on KoNViD-1k, a subjectively annotated VQA database consisting of 1,200 publicdomain video sequences, fairly sampled from a large public video dataset, YFCC100m. We present the challenges and choices we have made in creating such a database aimed at 'in the wild' authentic distortions, depicting a wide variety of content.

show abstract

Section: Related Workmentioning

confidence: 90%

The Konstanz natural video database (KoNViD-1k)

Hosu

Hahn

Jenadeleh

et al. 2017

2017 Ninth International Conference on Quality of Multimedia Experience (QoMEX)

Self Cite

240

245

View full text Add to dashboard Cite

show abstract

“…In contrast to previous work, Men et al [17] introduced an NR-VQA method that was trained using a natural video quality database, KoNViD-1k [9], which consists of 1200 unique video sequences with authentic distortions. In particular, a video-level feature vector was compiled by combining multiple features, such as blurriness, colorfulness, contrast, and spatial and temporal information.…”

Section: Related and Previous Workmentioning

confidence: 99%

“…All methods were evaluated using fivefold cross-validation with 10 random train-validationtest split, and median PLCC and SROCC values are reported as proposed in [17] and [18]. The median PLCC and SROCC values of five baseline methods (Video BLIINDS [23], VIIDEO [20], Video CORNIA [37], FC Model [17], and STFC Model [18]) were measured by Men et al in [17] and [18]. On the other hand, the results of STS-MLP [38] and STS-SVR [38] were taken from their original publication because their authors also report on median PLCC and SROCC values using fivefold cross-validation with 10 random train-validation-test split.…”

Section: Comparison With the State Of The Artmentioning

confidence: 99%

“…As regards LIVE VQA, our method was outperformed by approximately 0.05 in PLCC and SROCC by the best algorithm. Please note that previous methods except for FC Model [17] and STFC Model [18] were trained on or optimized for artificially distorted video sequences. That is why the results on the two different databases can be radically different.…”

Section: Comparison With the State Of The Artmentioning

confidence: 99%

See 1 more Smart Citation

No-reference video quality assessment via pretrained CNN and LSTM networks

Varga

Szirányi

2019

SIViP

View full text Add to dashboard Cite

A general-purpose no-reference video quality assessment algorithm based on a long short-term memory (LSTM) network and a pretrained convolutional neural network (CNN) is introduced. Considering video sequences as a time series of deep features extracted with the help of a CNN, an LSTM network is trained to predict subjective quality scores. In contrast to previous methods, the resulting algorithm was trained on the recently published Konstanz Natural Video Quality Database (KoNViD-1k), which is the only publicly available database that contains sequences with authentic distortions. The results of experiments on KoNViD-1k demonstrate that the proposed method outperforms other state-of-the-art algorithms. Furthermore, these results are also confirmed using tests on the LIVE Video Quality Assessment Database, which consists of artificially distorted videos.Keywords No-reference video quality assessment · Long short-term memory · Convolutional neural network B Domonkos Varga

show abstract

“…In the feature aggregation aspect, most methods aggregate framelevel features to video-level features by averaging them over the temporal axis [8,18,[22][23][24]35]. Li et al [19] adopt a 1D convolutional neural network to aggregate the primary features for a time interval.…”

Section: Temporal Modelingmentioning

confidence: 99%

Quality Assessment of In-the-Wild Videos

Jiang

2019

Proceedings of the 27th ACM International Conference on Multimedia

233

245

View full text Add to dashboard Cite

Quality assessment of in-the-wild videos is a challenging problem because of the absence of reference videos and shooting distortions. Knowledge of the human visual system can help establish methods for objective quality assessment of in-the-wild videos. In this work, we show two eminent effects of the human visual system, namely, content-dependency and temporal-memory effects, could be used for this purpose. We propose an objective no-reference video quality assessment method by integrating both effects into a deep neural network. For content-dependency, we extract features from a pre-trained image classification neural network for its inherent content-aware property. For temporal-memory effects, long-term dependencies, especially the temporal hysteresis, are integrated into the network with a gated recurrent unit and a subjectivelyinspired temporal pooling layer. To validate the performance of our method, experiments are conducted on three publicly available inthe-wild video quality assessment databases: KoNViD-1k, CVD2014, and LIVE-Qualcomm, respectively. Experimental results demonstrate that our proposed method outperforms five state-of-the-art methods by a large margin, specifically, 12.39%, 15.71%, 15.45%, and 18.09% overall performance improvements over the second-best method VBLIINDS, in terms of SROCC, KROCC, PLCC and RMSE, respectively. Moreover, the ablation study verifies the crucial role of both the content-aware features and the modeling of temporalmemory effects. The PyTorch implementation of our method is released at https://github.com/lidq92/VSFA. KEYWORDSvideo quality assessment; human visual system; content dependency; temporal-memory effects; in-the-wild videos ACM Reference Format:

show abstract

Empirical evaluation of no-reference VQA methods on a natural video quality database

Cited by 15 publications

References 8 publications

The Konstanz natural video database (KoNViD-1k)

The Konstanz natural video database (KoNViD-1k)

No-reference video quality assessment via pretrained CNN and LSTM networks

Quality Assessment of In-the-Wild Videos

Contact Info

Product

Resources

About