A Vocoder Based Method for Singing Voice Extraction

Abstract

This paper presents a novel method for extracting the vocal track from a musical mixture. The musical mixture consists of a singing voice and a backing track which may comprise of various instruments. We use a convolutional network with skip and residual connections as well as dilated convolutions to estimate vocoder parameters, given the spectrogram of an input mixture. The estimated parameters are then used to synthesize the vocal track, without any interference from the backing track. We evaluate our system, through objective metrics pertinent to audio quality and interference from background sources, and via a comparative subjective evaluation. We use open-source source separation systems based on Non-negative Matrix Factorization (NMFs) and Deep Learning methods as benchmarks for our system and discuss future applications for this particular algorithm.

Content Based Singing Voice Extraction from a Musical Mixture

“…We also observe that all three vocoder based models outperform the mask based model, UNET on isolating the vocal signal from the mixture. This follows from our previous observation [13] and is due to the approach that we follow of re-synthesising the vocal signal from the mixture rather than using a mask over the input signal. The models still lag behind on audio quality.…”

“…We use this prediction along with the vocoder features to synthesise the audio signal. We tried both the discrete representation of the fundamental frequency as described in [23] and a continuous representation, normalised to the range 0 to 1 as used in [13] and found that while the discrete representation leads to slightly higher accuracy in the output, the continuous representation produces a pitch contour perceptually more suitable for synthesis of the signal. Fig.…”

“…Instead we use a subjective listening test for evaluation [27]. We evaluate both the singer dependent network, SDN and the singer independent network, SIN, against a baseline [7], referred to as UNET and against our previously proposed model [13], SS. We adjusted the U-Net architecture to account for the change in the sampling frequency and the window size used in our model.…”

“…The short time fourier transform (STFT) was calculated with a Hanning window of size 1024. Both the STFT and the vocoder features were calculated with a hoptime of 5 ms. We use dimensionality reduction for the vocoder features [24,25,13], leading to 64 features.…”

“…The stem is the raw audio signal with linear and non-linear effects such as reverb, compression, delay and equalization among others. We recently proposed a model [13], which resynthesises the underlying raw vocal signal present in a musical signal by using a deep neural network architecture as a function approximator that predicts vocoder features pertaining to the vocal signal. The model outperformed state-of-theart source separation algorithms in terms of isolation of the signal from the backing track.…”