Multi-Channel Audio Source Separation Using Multiple Deformed References

Abstract

We present a general multi-channel source separation framework where additional audio references are available for one (or more) source(s) of a given mixture. Each audio reference is another mixture which is supposed to contain at least one source similar to one of the target sources. Deformations between the sources of interest and their references are modeled in a linear manner using a generic formulation. This is done by adding transformation matrices to an excitation-filter model, hence affecting different axes, namely frequency, dictionary component or time. A nonnegative matrix co-factorization algorithm and a generalized expectation-maximization algorithm are used to estimate the parameters of the model. Different model parameterizations and different combinations of algorithms are tested on music plus voice mixtures guided by music and/or voice references and on professionally-produced music recordings guided by cover references. Our algorithms improve the signal-to-distortion ratio (SDR) of the sources with the lowest intensity by 9 to 15 decibels (dB) with respect to original mixtures. Index Terms-Generalized Expectation-Maximization (GEM) algorithm, source separation. 2329-9290

On-the-Fly Audio Source Separation—A Novel User-Friendly Framework

“…and one can easily see that, while the new penalty keeps the group sparsity property thanks to Ψ gr (H) defined in (12), it prevents (when γ j > 0) the supergroups from vanishing since if H (j) 1 tends to zero, then − log H (j) 1 tends to +∞. This formulation generalizes the group sparsity constraint in the sense that (13) reduces to (12) for γ j = 0.…”

“…For the separation to be feasible, we require that every learned source model has a corresponding non-zero activation; however, this constraint is not enforced by the group sparsity penalty in (12) where it can happen that a group of different sources are fit together using the same source model, instead of separately using their designated models, rendering their separation impossible. We observed this "source vanishing" phenomenon in practice as illustrated in Fig.…”

“…This is due to the fact that W b and H b are now fully unconstrained in (11), whereas W is fixed and H is constrained by the group sparsity-inducing penalty. It can also be seen that increasing the trade-off parameters λ j in the penalty (12) (thus decreasing the number of active groups) increases the chances of source vanishing in both the supervised and semisupervised cases. In this section, we present a novel sparsityinducing penalty which helps prevent this problem completely.…”

“…Q. K. Duong synthesized signals (that may also include cover tracks as in [11]) called deformed references in [12] can be used to roughly learn the spectral and temporal characteristics of one or more sources in the mixtures so as to guide the separation process [8]- [10], [12]. A subclass of informed source separation approaches is user-guided separation methods where the prior information is provided by a user.…”

“…For this purpose, supervised approaches based on e.g., nonnegative matrix factorization (NMF) [20], [21] or its probabilistic formulation known as Probabilistic Latent Component Analysis (PLCA) [13], [22], where retrieved examples can be used to pre-learn the spectral dictionaries of the corresponding sources, are of great interest. Other methods in the prior art that couple the decomposition of the reference signals together with the mixture could also be considered [8], [11], [12], [23]. Regardless of the approach, several challenges, as detailed in Section II, arise in this on-the-fly framework due to (i) the unknown quality of the retrieved examples and (ii) possible lack of source descriptions (i.e.…”