Unsupervised Training for Deep Speech Source Separation with Kullback-Leibler Divergence Based Probabilistic Loss Function

Abstract

In this paper, we propose a multi-channel speech source separation with a deep neural network (DNN) which is trained under the condition that no clean signal is available. As an alternative to a clean signal, the proposed method adopts an estimated speech signal by an unsupervised speech source separation with a statistical model. As a statistical model of microphone input signal, we adopts a timevarying spatial covariance matrix (SCM) model which includes reverberation and background noise submodels so as to achieve robustness against reverberation and background noise. The DNN infers intermediate variables which are needed for constructing the timevarying SCM. Speech source separation is performed in a probabilistic manner so as to avoid overfitting to separation error. Since there are multiple intermediate variables, a loss function which evaluates a single intermediate variable is not applicable. Instead, the proposed method adopts a loss function which evaluates the output probabilistic signal directly based on Kullback-Leibler Divergence (KLD). Gradient of the loss function can be back-propagated into the DNN through all the intermediate variables. Experimental results under reverberant conditions show that the proposed method can train the DNN efficiently even when the number of training utterances is small, i.e., 1K.

Neural Full-Rank Spatial Covariance Analysis for Blind Source Separation

“…The source signals estimated by a conventional BSS method can be used as pseudo-supervised data. Togami et al [23] proposed to train a network to predict a multichannel Wiener filter (MWF) estimated by FCA. Drude et al [22] proposed to train a network by directly maximizing a log-marginal likelihood of a BSS model called complex angular central Gaussian mixture model (cACGMM) [31].…”

“…The mixture signals are generated at 16 kHz. As in the previous studies [22], [23], we performed dereverberation [35] to the mixture signals. For stabilizing the dereverberation, we added white Gaussian noise with the signal-to-noise ratio of 30 dB to the mixture signals.…”

“…Because the cACGMM and FCA cannot resolve the frequency permutation by themselves, we utilized an external permutation solver 1 . As unsupervised neural methods, we evaluated the pseudo supervised method [23] that approximates the results of the FCA [8] and the direct training method [22] that maximizes the likelihood of the cACGMM (neural cACGMM). The network outputs of these two methods were refined to fit the observation by the FCA and cACGMM.…”

“…Several unsupervised methods have been proposed to train a separation network by using multichannel mixture signals and the conventional statistical models [21]- [26]. The existing methods, however, resolve the frequency permutation ambiguity by using external permutation solvers [21]- [23] or the microphone array geometries [24]- [26], which limit the separation performance and/or their applicability. Our neural FCA, in contrast, resolves the permutation ambiguity by encouraging the independence of latent source features.…”

Surrogate Source Model Learning for Determined Source Separation

“…Recently, several works propose to learn the network parameters by directly optimizing the output of the separation [9]. They extend the method to unsupervised learning [10] and resource constrained environments [11]. We note that these latter methods make use of powerful, yet computation-hungry, spatial filtering techniques, limiting the number of iterations of the algorithms through which backpropagation can be safely done.…”