From Blind to Guided Audio Source Separation: How models and side information can improve the separation of sound

Abstract

Audio is a domain where signal separation has long been considered as a fascinating objective, potentially offering a wide range of new possibilities and experiences in professional and personal contexts, by better taking advantage of audio material and finely analyzing complex acoustic scenes. It has thus always been a major area for research in signal separation and an exciting challenge for industrial applications. Starting with blind separation of toy mixtures in the mid 90's, research has progressed up to real-world scenarios today, with applications to speech enhancement and recognition, music editing, 3D sound rendering, and audio information retrieval, among others. This has mostly been made possible by the development of increasingly informed separation techniques incorporating knowledge about the sources and/or the mixtures at hand. For instance, speech source separation for remote conferencing can benefit from prior knowledge of the room geometry and/or the names of the speakers, while music remastering will exploit instrument characteristics and knowledge of sound engineers mixing habits. After a brief historical account, we provide an overview of recent and ongoing research in this field, illustrating a variety of models and techniques designed so as to guide the audio source separation process towards efficient and robust solutions.

Separating time-frequency sources from time-domain convolutive mixtures using non-negative matrix factorization

“…We refer the reader to our web page for listening to audio examples illustrating the results discussed below 1 . Matlab code implementing the proposed method is also available.…”

“…Indeed, model-based approaches can take advantage of the very particular structure of audio signals in the TF plane [1]. For example, sparse component analysis methods [2] exploit the sparsity of the source signals in the TF domain.…”

Clustered multi-channel dereverberation for ad-hoc microphone arrays

“…In some recent research it is shown that reverberation can be exploited to obtain information about the recording environment such as room geometry and source locations [8,9,10,11]. Inspired by those works and given that clustered and informed approaches are shown to yield better results in ad-hoc microphone array contexts [1,2,6,12], this research is trying to find a setup independent approach to choose a subset of microphones that yields higher quality outputs in terms of echo cancellation. In order to achieve this goal the first step is to extract discriminative features from speech signals to choose a subset of single microphones (Section III.A).…”

Scalable audio separation with light Kernel Additive Modelling

“…It is a topic that has many applications in the entertainment industry such as automatic karaoke [19], [29], music upmixing [21], [22], [23] or audio restoration [31]. For this reason, it has gathered the attention of a large community of researchers in the past 15 years [35], [34]. The inherent difficulty of audio source separation comes from the fact that it is essentially an ill-posed inverse problem: in a typical setting, we have more signals to estimate than the number of signals we observe.…”

“…This track of research increasingly showed that prior assumptions about the sources leads to improved performance in practice [33], [16], [34]. The common ground of most related work on audio source separation then becomes the building of models for the spectrograms of the sources that have a strong expressive power while requiring the fitting of only a small number of parameters.…”