Spatiotemporal Feature Combination Model for No-Reference Video Quality Assessment

Men, Hui; Saupe, Dietmar

doi:10.1109/qomex.2018.8463426

Cited by 25 publications

(24 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The video-level feature vectors were mapped to subjective quality scores with an SVR. Later, this model was developed significantly [18] by combining spatial and temporal information more intensively.…”

Section: Related and Previous Workmentioning

confidence: 99%

“…First, we evaluated the design choices of our proposed method on KoNViD-1k [9], before comparing it with other state-of-the-art NR-VQA techniques. We evaluated our algorithm using fivefold cross-validation and report on median PLCC and SROCC values like Men et al [18] and Yan et al [38]. First of all, the effects of the applied pretrained CNNs and transfer learning were evaluated.…”

Section: Parameter Studymentioning

confidence: 99%

“…All methods were evaluated using fivefold cross-validation with 10 random train-validationtest split, and median PLCC and SROCC values are reported as proposed in [17] and [18]. The median PLCC and SROCC values of five baseline methods (Video BLIINDS [23], VIIDEO [20], Video CORNIA [37], FC Model [17], and STFC Model [18]) were measured by Men et al in [17] and [18]. On the other hand, the results of STS-MLP [38] and STS-SVR [38] were taken from their original publication because their authors also report on median PLCC and SROCC values using fivefold cross-validation with 10 random train-validation-test split.…”

Section: Comparison With the State Of The Artmentioning

confidence: 99%

“…As regards LIVE VQA, our method was outperformed by approximately 0.05 in PLCC and SROCC by the best algorithm. Please note that previous methods except for FC Model [17] and STFC Model [18] were trained on or optimized for artificially distorted video sequences. That is why the results on the two different databases can be radically different.…”

Section: Comparison With the State Of The Artmentioning

confidence: 99%

See 3 more Smart Citations

No-reference video quality assessment via pretrained CNN and LSTM networks

Varga

Szirányi

2019

SIViP

View full text Add to dashboard Cite

A general-purpose no-reference video quality assessment algorithm based on a long short-term memory (LSTM) network and a pretrained convolutional neural network (CNN) is introduced. Considering video sequences as a time series of deep features extracted with the help of a CNN, an LSTM network is trained to predict subjective quality scores. In contrast to previous methods, the resulting algorithm was trained on the recently published Konstanz Natural Video Quality Database (KoNViD-1k), which is the only publicly available database that contains sequences with authentic distortions. The results of experiments on KoNViD-1k demonstrate that the proposed method outperforms other state-of-the-art algorithms. Furthermore, these results are also confirmed using tests on the LIVE Video Quality Assessment Database, which consists of artificially distorted videos.Keywords No-reference video quality assessment · Long short-term memory · Convolutional neural network B Domonkos Varga

show abstract

Section: Related and Previous Workmentioning

confidence: 99%

Section: Parameter Studymentioning

confidence: 99%

Section: Comparison With the State Of The Artmentioning

confidence: 99%

Section: Comparison With the State Of The Artmentioning

confidence: 99%

See 2 more Smart Citations

No-reference video quality assessment via pretrained CNN and LSTM networks

Varga

Szirányi

2019

SIViP

View full text Add to dashboard Cite

show abstract

“…In the feature aggregation aspect, most methods aggregate framelevel features to video-level features by averaging them over the temporal axis [8,18,[22][23][24]35]. Li et al [19] adopt a 1D convolutional neural network to aggregate the primary features for a time interval.…”

Section: Temporal Modelingmentioning

confidence: 99%

Quality Assessment of In-the-Wild Videos

Jiang

2019

Proceedings of the 27th ACM International Conference on Multimedia

233

245

View full text Add to dashboard Cite

Quality assessment of in-the-wild videos is a challenging problem because of the absence of reference videos and shooting distortions. Knowledge of the human visual system can help establish methods for objective quality assessment of in-the-wild videos. In this work, we show two eminent effects of the human visual system, namely, content-dependency and temporal-memory effects, could be used for this purpose. We propose an objective no-reference video quality assessment method by integrating both effects into a deep neural network. For content-dependency, we extract features from a pre-trained image classification neural network for its inherent content-aware property. For temporal-memory effects, long-term dependencies, especially the temporal hysteresis, are integrated into the network with a gated recurrent unit and a subjectivelyinspired temporal pooling layer. To validate the performance of our method, experiments are conducted on three publicly available inthe-wild video quality assessment databases: KoNViD-1k, CVD2014, and LIVE-Qualcomm, respectively. Experimental results demonstrate that our proposed method outperforms five state-of-the-art methods by a large margin, specifically, 12.39%, 15.71%, 15.45%, and 18.09% overall performance improvements over the second-best method VBLIINDS, in terms of SROCC, KROCC, PLCC and RMSE, respectively. Moreover, the ablation study verifies the crucial role of both the content-aware features and the modeling of temporalmemory effects. The PyTorch implementation of our method is released at https://github.com/lidq92/VSFA. KEYWORDSvideo quality assessment; human visual system; content dependency; temporal-memory effects; in-the-wild videos ACM Reference Format:

show abstract