Music Source Activity Detection and Separation Using Deep Attractor Network

Abstract

In music signal processing, singing voice detection and music source separation are widely researched topics. Recent progress in deep neural network based source separation has advanced the state of the performance in the problem of vocal and instrument separation, while the problem of joint source activity detection and separation remains unexplored. In this paper, we propose an approach to perform source activity detection using the high-dimensional embedding generated by Deep Attractor Network (DANet) when trained for music source separation. By defining both tasks together, DANet is able to dynamically estimate the number of outputs depending on the active sources. We propose an Expectation-Maximization (EM) training paradigm for DANet which further improves the separation performance of the original DANet. Experiments show that our network achieves higher source separation and comparable source activity detection against a baseline system.

Class-conditional Embeddings for Music Source Separation

“…Recently, the work in [17] demonstrated that a common embedding space for musical instrument separation using various deep attractor networks could achieve competitive performance. Our system is similar to the anchored and/or expectation-maximization deep attractor networks in [17], but we use an auxiliary network to estimate the mean and covariance parameters for each instrument. We also explore what type of covariance model is most effective for musical source separation (tied vs. untied across classes, diagonal vs. spherical).…”

Bootstrapping Single-channel Source Separation via Unsupervised Spatial Clustering on Stereo Mixtures

“…The model we use for source separation is based on deep clustering (DC) [6,21]. We selected deep clustering because it is a highly successful approach that has inspired multiple successful variants [22][23][24][25][26][27]. Further, its separation framework is somewhat connected to our primitive spatial separation as it is based on clustering as well, but in a learned embedding space, and its objective function has been shown to be amenable to the introduction of weights.…”

Improving Universal Sound Separation Using Sound Classification

“…Specifically for separation tasks, speaker-discriminative embeddings are produced for targeted voice separation in [6] and for diarization in [17] yielding a significant improvement over the unconditional separation framework. Recent works [18,19] have utilized conditional embeddings for each music class in order to boost the performance of a deep attractor-network [20] for music separation.…”