Audio Time Stretching Using Fuzzy Classification of Spectral Bins

Abstract

A novel method for audio time stretching has been developed. In time stretching, the audio signal's duration is expanded, whereas its frequency content remains unchanged. The proposed time stretching method employs the new concept of fuzzy classification of time-frequency points, or bins, in the spectrogram of the signal. Each time-frequency bin is assigned, using a continuous membership function, to three signal classes: tonalness, noisiness, and transientness. The method does not require the signal to be explicitly decomposed into different components, but instead, the computing of phase propagation, which is required for time stretching, is handled differently in each time-frequency point according to the fuzzy membership values. The new method is compared with three previous time-stretching methods by means of a listening test. The test results show that the proposed method yields slightly better sound quality for large stretching factors as compared to a state-of-the-art algorithm, and practically the same quality as a commercial algorithm. The sound quality of all tested methods is dependent on the audio signal type. According to this study, the proposed method performs well on music signals consisting of mixed tonal, noisy, and transient components, such as singing, techno music, and a jazz recording containing vocals. It performs less well on music containing only noisy and transient sounds, such as a drum solo. The proposed method is applicable to the high-quality time stretching of a wide variety of music signals.

A time-scale modification dataset with subjective quality labels

“…The difference in quality between β < 1 and β > 1 was mentioned briefly by Sylvestre and Kabal (1992) and shown in early results from this testing in (Roberts and Paliwal, 2019). Since the release of the MATLAB TSM Toolbox , the included algorithms, PV, IPL, WSOLA and HPTSM, have been used in most evaluations, while comparisons to commercial algorithms are rare (Damskägg and Välimäki, 2017;Karrer et al, 2006). The source audio used during testing also varies between papers with some papers using the files provided with the MATLAB TSM Toolbox.…”

“…Fuzzy classification of spectral bins (FuzzyPV) (Damskägg and Välimäki, 2017), is an extension of the IPL. Spectral bins are given a degree of membership to three classes, sinusoidal, noise and transient, resulting in a fuzzy classification of each bin.…”

Special Issue on “Sound and Music Computing”

“…Damskägg and Välimäki [4] address a problem known as time-scale modification in which the objective is to temporally stretch or compress a given audio signal while preserving properties like pitch and timbre. To handle different signal characteristics, the main idea of the paper is to modify the phase of the signal's time-frequency bins in an adaptive fashion using an implicit bin-wise fuzzy classification based on three classes (sinusoid, noise, transient).…”

An Objective Measure of Quality for Time-Scale Modification of Audio

“…The testing subset, containing 240 files, was created using three additional methods to time-scale 20 reference files at a random β in each band of 0.25 < β < 0.5, 0.5 < β < 0.8, 0.8 < β < 1 and 1 < β < 2. Elastique by Zplane Development, the Phase Vocoder using fuzzy classification of bins (FuzzyPV) by Damskägg and Välimäki (2017) and Non-Negative Matrix Factorisation Time-Scale Modification (NMFTSM) by Roma et al (2019) were used to generate the testing subset. Finally, an evaluation subset was generated by processing the testing subset reference files with all previously mentioned methods, in addition to the Scaled Phase-Locking Phase Vocoder (SPL) by Laroche and Dolson (1999), IPL and SPL variants of PhaVoRIT (IPL and SPL) by Karrer et al (2006) and Epoch Synchronous Overlap-Add (ESOLA) by Rudresh et al (2018).…”