A time-scale modification dataset with subjective quality labels

Abstract

Time Scale Modification (TSM) is a well-researched field; however, no effective objective measure of quality exists. This paper details the creation, subjective evaluation, and analysis of a dataset for use in the development of an objective measure of quality for TSM. Comprised of two parts, the training component contains 88 source files processed using six TSM methods at 10 time scales, while the testing component contains 20 source files processed using three additional methods at four time scales. The source material contains speech, solo harmonic and percussive instruments, sound effects, and a range of music genres. Ratings (42 529) were collected from 633 sessions using laboratory and remote collection methods. Analysis of results shows no correlation between age and quality of rating; expert and non-expert listeners to be equivalent; minor differences between participants with and without hearing issues; and minimal differences between testing modalities. A comparison of published objective measures and subjective scores shows the objective measures to be poor indicators of subjective quality. Initial results for a retrained objective measure of quality are presented with results approaching average root mean squared error loss and Pearson correlation values of subjective sessions. The labeled dataset is available at http://ieee-dataport.org/1987.

An objective measure of quality for time-scale modification of audio

“…The Pearson Correlation Coefficient (PCC; q) and L were used as network performance measures. The composite measure of Roberts and Paliwal (2020) was used when selecting the ideal epoch after training. The optimal epoch was chosen as the epoch with the minimum overall distance (D), calculated by…”

“…Network hyper-parameters were optimized through a systematic non-exhaustive search. Each method of alignment was trained to the SMOS, MedianOS, raw SMOS, and raw MedianOS targets, and raw values were calculated prior to subjective session normalization in Roberts and Paliwal (2020). Additionally, baseline conditions, the inclusion of reference files within the training set, concatenation of logarithmic transforms of features, and combinations of multiple alignment methods were considered.…”

“…In this work, we propose the first objective measure of quality (OMOQ) for time-scale modified audio. It uses handcrafted features with deep-learning methods and is trained using a recently published dataset (Roberts, 2020). The contributions of this paper are an OMOQ for time-scaled audio, novel quality features specific to TSM and comparison of TSM methods using the objective measure.…”

“…Training of deep-learning methods requires a large amount of labelled signals. Recently, a dataset of time-scaled audio with subjective labels was published for this purpose (Roberts and Paliwal, 2020). Reference files were drawn from a large variety of sources, including speech, singing, solo harmonic, and percussive instruments, as well as a variety of musical genres.…”

“…Subjective median opinion scores (MedianOS) and subjective mean opinion scores (SMOS) before and after normalization were provided as labels. The dataset was published under the Creative Commons Attribution 4.0 International (CC BY 4.0, Mountain View, CA) license and is available online 1 (Roberts, 2020).…”