Scalable audio separation with light Kernel Additive Modelling

Abstract

Recently, Kernel Additive Modelling (KAM) was proposed as a unified framework to achieve multichannel audio source separation. Its main feature is to use kernel models for locally describing the spectrograms of the sources. Such kernels can capture source features such as repetitivity, stability over time and/or frequency, self-similarity, etc. KAM notably subsumes many popular and effective methods from the state of the art, including REPET and harmonic/percussive separation with median filters. However, it also comes with an important drawback in its initial form: its memory usage badly scales with the number of sources. Indeed, KAM requires the storage of the full-resolution spectrogram for each source, which may become prohibitive for full-length tracks or many sources. In this paper, we show how it can be combined with a fast compression algorithm of its parameters to address the scalability issue, thus enabling its use on small platforms or mobile devices.

An analysis of environment, microphone and data simulation mismatches in robust speech recognition

“…In addition, training on real data now leads to a performance decrease on simulated data, while Sivasankaran et al (2015) found it to consistently improve performance on both real and simulated data. Along with the recent results of Nugraha et al (2016b) on another dataset, this suggests that, although weighted EM made little difference for spectral models other than DNN (Liutkus et al, 2015), weighted EM outperforms exact EM for the estimation of multichannel statistics from DNN outputs. More work on the estimation of multichannel statistics from DNN outputs is therefore also required.…”

“…We performed this experiment using the DNN-based multichannel source separation technique of Nugraha et al (2016a), which is a variant of the one of Sivasankaran et al (2015) that relies on exact EM updates for the spatial covariance matrices (Duong et al, 2010) instead of the weighted EM updates of Liutkus et al (2015).…”

Multi-Scale multi-band densenets for audio source separation

“…Audio source separation has attracted considerable attention in the last decade. Various approaches have been introduced so far such as local Gaussian modeling [1,2], non-negative factorization [3][4][5], kernel additive modeling [6] and combinations of those approaches [7][8][9]. Recently, deep neural networks (DNNs) based source separation methods has shown significant improvement in separation performance over earlier methods.…”

Adversarial Semi-Supervised Audio Source Separation Applied to Singing Voice Extraction

“…We use the training partition of the DSD100 [9] database as our supervised training set Dm. We split the multi-track databases iKala [7], MedleyDB [19] and CCMixter [20] into thirds, and use one third of tracks from each database to form the unlabelled dataset Du and the source datasets D k s needed for semi-supervised training. Our validation and test set is each built by taking another third of tracks from iKala, MedleyDB, and CCMixter, in addition to 25 tracks from the test partition of DSD100.…”