A recurrent encoder-decoder approach with skip-filtering connections for monaural singing voice separation

Abstract

The objective of deep learning methods based on encoder-decoder architectures for music source separation is to approximate either ideal time-frequency masks or spectral representations of the target music source(s). The spectral representations are then used to derive time-frequency masks. In this work we introduce a method to directly learn time-frequency masks from an observed mixture magnitude spectrum. We employ recurrent neural networks and train them using prior knowledge only for the magnitude spectrum of the target source. To assess the performance of the proposed method, we focus on the task of singing voice separation. The results from an objective evaluation show that our proposed method provides comparable results to deep learning based methods which operate over complicated signal representations. Compared to previous methods that approximate time-frequency masks, our method has increased performance of signal to distortion ratio by an average of 3.8 dB.

Reducing Interference with Phase Recovery in DNN-based Monaural Singing Voice Separation

“…2. The Masker consists of a bi-directional recurrent neural network (Bi-RNN), the RNN encoder (RNNenc), an RNN decoder (RNNdec), a sparsifying transform that is implemented by a feed-forward neural network (FNN), with shared weights through time, followed by a rectified linear unit (ReLU), and the skip-filtering connections [16]. The input to the Masker is a Vx and the output of the skip-filtering connections is a first estimate of the singing voice spectrogram denotedV 1 .…”

“…SinceV 1 is expected to contain interference from other music sources [16,17], the Denoiser aims at further enhancing the estimate of the Masker. A denoising filter is learned and applied to the estimate of the Masker,V 1 .…”

“…Indeed, state-of-the-art results for source separation are obtained with deep learning methods in both monaural [12,13] and multichannel [14,15] scenarios. This goes for the particular case of monaural singing voice separation [16,17,18]. The most recent approach, which is called MaD TwinNet [18], predicts a voice magnitude spectrogram that is further combined with the mixture's phase.…”

Harmonic-Percussive Source Separation with Deep Neural Networks and Phase Recovery

“…where is the Hadamard product. V 1 is expected to contain interferences from other music sources [17,18]. Therefore, MaD utilizes another module, the Denoiser, on top of the Masker, which consists of two feed-forward layers denoted FNNenc and FNNdec.…”

“…However, state-of-the-art results for source separation are obtained with deep learning methods [15,16], which learn the model from the given data. They have shown particularly successful for the task of singing voice separation [17,18,19]. Recently, a DNN-based HPSS method has been introduced [20] and is based on learning a set of convolution kernels that perform the separation.…”