Flow-Based Independent Vector Analysis for Blind Source Separation

Bando

IEEE/ACM Trans. Audio Speech Lang. Process.

et al. 2022

Self Cite

This paper describes a computationally-efficient statistical approach to joint (semi-)blind source separation and dereverberation for multichannel noisy reverberant mixture signals.A standard approach to source separation is to formulate a generative model of a multichannel mixture spectrogram that consists of source and spatial models representing the time-frequency power spectral densities (PSDs) and spatial covariance matrices (SCMs) of source images, respectively, and find the maximumlikelihood estimates of these parameters. A state-of-the-art blind source separation method in this thread of research is fast multichannel nonnegative matrix factorization (FastMNMF) based on the low-rank PSDs and jointly-diagonalizable full-rank SCMs. To perform mutually-dependent separation and dereverberation jointly, in this paper we integrate both moving average (MA) and autoregressive (AR) models that represent the early reflections and late reverberations of sources, respectively, into the Fast-MNMF formalism. Using a pretrained deep generative model of speech PSDs as a source model, we realize semi-blind joint speech separation and dereverberation. We derive an iterative optimization algorithm based on iterative projection or iterative source steering for jointly and efficiently updating the AR parameters and the SCMs. Our experimental results showed the superiority of the proposed ARMA extension over its AR-or MA-ablated version in a speech separation and/or dereverberation task.

Section: Discussionmentioning

confidence: 99%

Autoregressive Moving Average Jointly-Diagonalizable Spatial Covariance Analysis for Joint Source Separation and Dereverberation

Bando

IEEE/ACM Trans. Audio Speech Lang. Process.

et al. 2022

Self Cite

“…Considering the recent advance of deep learning techniques, one important future direction is to use a normalizing flow [52] for formulating an adaptive time-varying spatial model as proposed in [53]. Another complementary direction is to use a deep generative model of speech for improving the expression capability of the source model as proposed in [54], [55].…”

Section: Speech Separation With (Over)determined Configurationsmentioning

confidence: 99%

Generalized Fast Multichannel Nonnegative Matrix Factorization Based on Gaussian Scale Mixtures for Blind Source Separation

Fontaine

IEEE/ACM Trans. Audio Speech Lang. Process.

et al. 2022

Self Cite

This paper describes heavy-tailed extensions of a state-of-the-art versatile blind source separation method called fast multichannel nonnegative matrix factorization (FastMNMF) from a unified point of view. The common way of deriving such an extension is to replace the multivariate complex Gaussian distribution in the likelihood function with its heavy-tailed generalization, e.g., the multivariate complex Student's t and leptokurtic generalized Gaussian distributions, and tailor-make the corresponding parameter optimization algorithm. Using a wider class of heavy-tailed distributions called a Gaussian scale mixture (GSM), i.e., a mixture of Gaussian distributions whose variances are perturbed by positive random scalars called impulse variables, we propose GSM-FastMNMF and develop an expectationmaximization algorithm that works even when the probability density function of the impulse variables have no analytical expressions. We show that existing heavy-tailed FastMNMF extensions are instances of GSM-FastMNMF and derive a new instance based on the generalized hyperbolic distribution that include the normal-inverse Gaussian, Student's t, and Gaussian distributions as the special cases. Our experiments show that the normalinverse Gaussian FastMNMF outperforms the state-of-the-art FastMNMF extensions and ILRMA model in speech enhancement and separation in terms of the signal-to-distortion ratio.

“…Besides supervised separation methods based on deep neural networks (DNNs) [3][4][5] that have been shown to work well, there is an increasing interest in DNN-based methods for semi-supervised separation and unsupervised separation, a.k.a. blind source separation (BSS), because of their potential in handling unseen sources in unknown environments [6][7][8][9][10].…”

Section: Introductionmentioning

confidence: 99%

“…Independent vector analysis (IVA) [12,13] is a classical BSS technique for determined separation case that decomposes M mixture STFT spectra (obtained from an M -channel recording) into spectra of N sources (N = M ) using time-invariant demixing matrices. By contrast, NF-IVA [8] uses timevarying demixing matrices represented by a normalizing flow (NF) [14]. It includes multilayer perceptrons (MLPs) that are optimized from scratch at run-time with backpropagation (BP) [15] given only the observed mixture.…”

Section: Introductionmentioning

confidence: 99%

Flow-Based Fast Multichannel Nonnegative Matrix Factorization for Blind Source Separation

ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Fontaine

et al. 2022

Self Cite

This paper describes a blind source separation method for multichannel audio signals, called NF-FastMNMF, based on the integration of the normalizing flow (NF) into the multichannel nonnegative matrix factorization with jointly-diagonalizable spatial covariance matrices, a.k.a. FastMNMF. Whereas the NF of flow-based independent vector analysis, called NF-IVA, acts as the demixing matrices to transform an M -channel mixture into M independent sources, the NF of NF-FastMNMF acts as the diagonalization matrices to transform an Mchannel mixture into a spatially-independent M -channel mixture represented as a weighted sum of N source images. This diagonalization enables the NF, which has been used only for determined separation because of its bijective nature, to be applicable to non-determined separation. NF-FastMNMF has time-varying diagonalization matrices that are potentially better at handling dynamical data variation than the time-invariant ones in FastMNMF. To have an NF with richer expression capability, the dimension-wise scalings using diagonal matrices originally used in NF-IVA are replaced with linear transformations using upper triangular matrices; in both cases, the diagonal and upper triangular matrices are estimated by neural networks. The evaluation shows that NF-FastMNMF performs well for both determined and non-determined separations of multiple speech utterances by stationary or non-stationary speakers from a noisy reverberant mixture.