Speaker-Independent Speech Separation With Deep Attractor Network

Abstract

Despite the recent success of deep learning for many speech processing tasks, single-microphone, speaker-independent speech separation remains challenging for two main reasons. The first reason is the arbitrary order of the target and masker speakers in the mixture (permutation problem), and the second is the unknown number of speakers in the mixture (output dimension problem). We propose a novel deep learning framework for speech separation that addresses both of these issues. We use a neural network to project the time-frequency representation of the mixture signal into a high-dimensional embedding space. A reference point (attractor) is created in the embedding space to represent each speaker which is defined as the centroid of the speaker in the embedding space. The time-frequency embeddings of each speaker are then forced to cluster around the corresponding attractor point which is used to determine the time-frequency assignment of the speaker. We propose three methods for finding the attractors for each source in the embedding space and compare their advantages and limitations. The objective function for the network is standard signal reconstruction error which enables end-to-end operation during both training and test phases. We evaluated our system using the Wall Street Journal dataset (WSJ0) on two and three speaker mixtures and report comparable or better performance than other state-of-the-art deep learning methods for speech separation.

Interrupted and Cascaded Permutation Invariant Training for Speech Separation

“…We also compare the results with those of prior works in Table 2, all data on the test set for comparison, including scale-invariant signal-to-noise ratio improvement (SI-SNR i ) [6] and SDR i in dB. Row (a) is for the baseline separation model TasNet-v2 [4] we used throughout this work.…”

“…Most of such approaches first transform the time-domain mixture waveform into some feature map, such as the spectrogram or 2-D feature map encoded by 1-D convolution blocks. An often used approach is then to infer a mask for each individual speaker [5,6,7,8,9,10], and multiply the masks element-wise with the mixture feature map to obtain the individual feature maps. A recent work integrating different mixture representations and performing cross-domain joint clustering for mask-inference has also shown encouraging improvements [11].…”

Improving Universal Sound Separation Using Sound Classification

“…Specifically for separation tasks, speaker-discriminative embeddings are produced for targeted voice separation in [6] and for diarization in [17] yielding a significant improvement over the unconditional separation framework. Recent works [18,19] have utilized conditional embeddings for each music class in order to boost the performance of a deep attractor-network [20] for music separation.…”

Unsupervised Training for Deep Speech Source Separation with Kullback-Leibler Divergence Based Probabilistic Loss Function

“…As supervised speech source separation techniques, recently, deep neural network (DNN) based approaches with a training dataset This work was done while Yoshiki Masuyama and Yu Nakagome were interns at LINE Corporation. in which there are microphone input signal and corresponding oracle clean data have been widely studied, e.g., deep clustering (DC) [10,11], permutation invariant training (PIT) [12,13], deep attractor network [14,15], and hybrid approaches with BSS [16][17][18]. DNN based approaches can capture complicated spectral characteristics of a speech source.…”