Unsupervised Deep Clustering for Source Separation: Direct Learning from Mixtures Using Spatial Information

Tzinis, Efthymios; Venkataramani, Shrikant; Smaragdis, Paris

doi:10.1109/icassp.2019.8683201

Cited by 35 publications

(28 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Multichannel-audio-based methods, on the other hand, can train a DNN to separate sound sources out of view or behind obstacles. Tzinis et al [14] trained a monaural separation network by using source signals estimated by applying K-means clustering on interchannel phase differences (IPDs) between two microphones. Almost simultaneously, Drude et al [12] proposed a similar approach that uses signals separated by the cACGMM [21].…”

Section: Unsupervised Training Of Neural Source Separationmentioning

confidence: 99%

See 1 more Smart Citation

Deep Bayesian Unsupervised Source Separation Based On A Complex Gaussian Mixture Model

Bando

Sasaki

Yoshii

2019

2019 IEEE 29th International Workshop on Machine Learning for Signal Processing (MLSP)

View full text Add to dashboard Cite

This paper presents an unsupervised method that trains neural source separation by using only multichannel mixture signals. Conventional neural separation methods require a lot of supervised data to achieve excellent performance. Although multichannel methods based on spatial information can work without such training data, they are often sensitive to parameter initialization and degraded with the sources located close to each other. The proposed method uses a cost function based on a spatial model called a complex Gaussian mixture model (cGMM). This model has the time-frequency (TF) masks and direction of arrivals (DoAs) of sources as latent variables and is used for training separation and localization networks that respectively estimate these variables. This joint training solves the frequency permutation ambiguity of the spatial model in a unified deep Bayesian framework. In addition, the pre-trained network can be used not only for conducting monaural separation but also for efficiently initializing a multichannel separation algorithm. Experimental results with simulated speech mixtures showed that our method outperformed a conventional initialization method.

show abstract

Section: Unsupervised Training Of Neural Source Separationmentioning

confidence: 99%

“…Unsupervised training for neural source separation using multichannel mixture signals has recently gained a lot of attention [12][13][14][15]. One approach is to generate supervised data by using multichannel separation methods [12][13][14]. This approach suffers from the estimation errors of the multichannel methods mentioned above.…”

Section: Introductionmentioning

confidence: 99%

Deep Bayesian Unsupervised Source Separation Based On A Complex Gaussian Mixture Model

Bando

Sasaki

Yoshii

2019

2019 IEEE 29th International Workshop on Machine Learning for Signal Processing (MLSP)

View full text Add to dashboard Cite

show abstract

“…Recently, unsupervised DNN training techniques have been proposed [19,20]. These techniques estimate a time-frequency mask based on the DC.…”

Section: Introductionmentioning

confidence: 99%

Unsupervised Training for Deep Speech Source Separation with Kullback-Leibler Divergence Based Probabilistic Loss Function

Togami

Masuyama

Komatsu

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

In this paper, we propose a multi-channel speech source separation with a deep neural network (DNN) which is trained under the condition that no clean signal is available. As an alternative to a clean signal, the proposed method adopts an estimated speech signal by an unsupervised speech source separation with a statistical model. As a statistical model of microphone input signal, we adopts a timevarying spatial covariance matrix (SCM) model which includes reverberation and background noise submodels so as to achieve robustness against reverberation and background noise. The DNN infers intermediate variables which are needed for constructing the timevarying SCM. Speech source separation is performed in a probabilistic manner so as to avoid overfitting to separation error. Since there are multiple intermediate variables, a loss function which evaluates a single intermediate variable is not applicable. Instead, the proposed method adopts a loss function which evaluates the output probabilistic signal directly based on Kullback-Leibler Divergence (KLD). Gradient of the loss function can be back-propagated into the DNN through all the intermediate variables. Experimental results under reverberant conditions show that the proposed method can train the DNN efficiently even when the number of training utterances is small, i.e., 1K.

show abstract

“…Another option is to generate intermediate masks with an unsupervised teacher, as proposed in e.g. [10,11], and also in [12] where we demonstrate how to leverage a probabilistic spatial mixture model, namely a complex angular central Gaussian mixture model (cACGMM), to generate intermediate masks. However, this approaches require a -possibly hand-crafted -teacher system and also a lot of computational resources to either store the intermediate masks or generate them on-the-fly.…”

Section: Introductionmentioning

confidence: 99%

Unsupervised Training of Neural Mask-Based Beamforming

2019

View full text Add to dashboard Cite

We present an unsupervised training approach for a neural network-based mask estimator in an acoustic beamforming application. The network is trained to maximize a likelihood criterion derived from a spatial mixture model of the observations. It is trained from scratch without requiring any parallel data consisting of degraded input and clean training targets. Thus, training can be carried out on real recordings of noisy speech rather than simulated ones. In contrast to previous work on unsupervised training of neural mask estimators, our approach avoids the need for a possibly pre-trained teacher model entirely. We demonstrate the effectiveness of our approach by speech recognition experiments on two different datasets: one mainly deteriorated by noise (CHiME 4) and one by reverberation (REVERB). The results show that the performance of the proposed system is on par with a supervised system using oracle target masks for training and with a system trained using a model-based teacher.

show abstract

Unsupervised Deep Clustering for Source Separation: Direct Learning from Mixtures Using Spatial Information

Cited by 35 publications

References 14 publications

Deep Bayesian Unsupervised Source Separation Based On A Complex Gaussian Mixture Model

Deep Bayesian Unsupervised Source Separation Based On A Complex Gaussian Mixture Model

Unsupervised Training for Deep Speech Source Separation with Kullback-Leibler Divergence Based Probabilistic Loss Function

Unsupervised Training of Neural Mask-Based Beamforming

Contact Info

Product

Resources

About