Subjective and objective evaluation of an audiovisual subjective dataset for research and development

Pinson, Margaret H.; Schmidmer, Christian; Janowski, Lucjan; Pépion, Romuald; Huynh-Thu, Quan; Corriveau, Philip; Younkin, Audrey; Callet, Patrick Le; Barkowsky, Marcus; Ingram, William A.

doi:10.1109/qomex.2013.6603199

Cited by 16 publications

(6 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…POLQA was designed for speech quality assessment and can be run in narrowband mode (telephone quality; 300-3400 Hz) or superwideband (SWB) mode (50-14 000 Hz). POLQA has been shown to have potential as an audio quality model 13 and the developers of POLQA are currently working on adapting it for use in audio quality evaluation. An alternative speech quality model called ViSQOL, 14 has been adapted for audio quality testing.…”

Section: Objective Metricsmentioning

confidence: 99%

ViSQOLAudio: An objective audio quality metric for low bitrate codecs

Hines

Gillen

Kelly

et al. 2015

The Journal of the Acoustical Society of America

View full text Add to dashboard Cite

Streaming services seek to optimise their use of bandwidth across audio and visual channels to maximise the quality of experience for users. This letter evaluates whether objective quality metrics can predict the audio quality for music encoded at low bitrates by comparing objective predictions with results from listener tests. Three objective metrics were benchmarked: PEAQ, POLQA, and VISQOLAudio. The results demonstrate objective metrics designed for speech quality assessment have a strong potential for quality assessment of low bitrate audio codecs.

show abstract

Section: Objective Metricsmentioning

confidence: 99%

ViSQOLAudio: An objective audio quality metric for low bitrate codecs

Hines

Gillen

Kelly

et al. 2015

The Journal of the Acoustical Society of America

View full text Add to dashboard Cite

show abstract

“…growing distribution). Recently published studies asking for influences based on different devices used or different (home) usage environments show, that for subjective testing with naїves it does not have that much influence on evaluation results compared to studies conducted within the laboratory (see [5], [6], and [10]). However, in some cases it still might be interesting how to define the home environment or several kinds of home environments.…”

Section: Discussion Of Resultsmentioning

confidence: 99%

Defining the Viewing Conditions in the Home Environment and its Influences

Kepplinger¹,

Jauch²,

Tobian³

2013

4th International Workshop on Perceptual Quality of Systems (PQS 2013)

View full text Add to dashboard Cite

This work describes the activities towards the definition of modern TV viewing environments and possible changes over time from a standardization point of view compared to the real usage in the living room. Up to now, standards are available in order to recommend the ideal viewing environment for different viewing conditions including TV system, usage purpose and kind of visualization (e.g. 2D vs. 3D) for subjective assessment of quality in laboratory environments. In order to define these and similar factors for the therein differentiated home environment a study was conducted investigating the collection of data which is relevant for future evaluation purposes. The results show, that different profiles might be detectable concerning room conditions but not concerning the illumination and ambient light conditions deriving from a small-scale field examination. Based on this, we would like to discuss the process of defining the home environment and its influence including ambient light conditions.

show abstract

“…Table III shows the same analysis, using subjective data instead of objective data. It performs lab-to-lab comparisons [31] 0.969 0.434 0.708 VQEG-MM2 [6] 0.965 0.350 0.833 ITS4S Everglades [26] 0.960 0.298 0.930 using experiments that were conducted by six or more international labs: the common set from VQEG-HDTV [31] and VQEG-MM2 [6], [33]. We made lab-to-lab pairwise comparisons and then picked the median over all pairs.…”

Section: Benchmarkmentioning

confidence: 99%

Subjective Assessment Experiments That Recruit Few Observers With Repetitions (FOWR)

Perez,

Janowski,

Garcia

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Recent studies have shown that it is possible to characterize subject bias and variance in subjective assessment tests. Apparent differences among subjects can, for the most part, be explained by random factors. Building on that theory, we propose a subjective test design where three to four team members each rate the stimuli multiple times. The results are comparable to a high performing objective metric. This provides a quick and simple way to analyze new technologies and perform pre-tests for subjective assessment.

show abstract

Subjective and objective evaluation of an audiovisual subjective dataset for research and development

Abstract: International audienceIn 2011, the Video Quality Experts Group (VQEG) ran subjects through the same audiovisual subjective test at six different international laboratories. That small dataset is now publically available for research and development purposes

Cited by 16 publications

References 3 publications

ViSQOLAudio: An objective audio quality metric for low bitrate codecs

ViSQOLAudio: An objective audio quality metric for low bitrate codecs

Defining the Viewing Conditions in the Home Environment and its Influences

Subjective Assessment Experiments That Recruit Few Observers With Repetitions (FOWR)

Contact Info

Product

Resources

About