Adversarial Semi-Supervised Audio Source Separation Applied to Singing Voice Extraction

Abstract

The state of the art in music source separation employs neural networks trained in a supervised fashion on multi-track databases to estimate the sources from a given mixture. With only few datasets available, often extensive data augmentation is used to combat overfitting. Mixing random tracks, however, can even reduce separation performance as instruments in real music are strongly correlated. The key concept in our approach is that source estimates of an optimal separator should be indistinguishable from real source signals. Based on this idea, we drive the separator towards outputs deemed as realistic by discriminator networks that are trained to tell apart real from separator samples. This way, we can also use unpaired source and mixture recordings without the drawbacks of creating unrealistic music mixtures. Our framework is widely applicable as it does not assume a specific network architecture or number of sources. To our knowledge, this is the first adoption of adversarial training for music source separation. In a prototype experiment for singing voice separation, separation performance increases with our approach compared to purely supervised training.

Jointly Detecting and Separating Singing Voice: A Multi-Task Approach

“…As a baseline system for SVS, we implemented a variant of the U-Net described in Section 3.2 and shown in Figure 1. The approach is similar to [19] and [11] and outputs a mask when given spectrogram magnitudes of a mixture excerpt. During training, audio excerpts are randomly selected from the multi-track dataset, and converted to a log-normalised spectrogram representation.…”

Semi-supervised Monaural Singing Voice Separation with a Masking Network Trained on Synthetic Mixtures

“…As in the unconditional setting, the discriminator attempts to differentiate between real and generated images. Adversarial training was used for supervised source separation, where the distribution of each of the mixture components is known and modeled by a GAN, by Stoller et al [11] and Subkhan et al [12]. The adversarial training was motivated as being better able to deal with correlated sources.…”

Snore-GANs: Improving Automatic Snore Sound Classification With Synthesized Data

“…The generation of sequential samples, however, heavily relies on the context information [24]. Albeit a handful of related studies reported in the audio processing domain, they either focus on speech enhancement [29], [30] or music creation [31].…”