Localization based stereo speech source separation using probabilistic time-frequency masking and deep neural networks

Abstract

Time-frequency (T-F) masking is an effective method for stereo speech source separation. However, reliable estimation of the T-F mask from sound mixtures is a challenging task, especially when room reverberations are present in the mixtures. In this paper, we propose a new stereo speech separation system where deep neural networks are used to generate soft T-F mask for separation. More specifically, the deep neural network, which is composed of two sparse autoencoders and a softmax regression, is used to estimate the orientations of the dominant source at each T-F unit, based on low-level features, such as mixing vector (MV), interaural level, and phase difference (IPD/ILD). The dataset for training the networks was generated by the convolution of binaural room impulse responses (RIRs) and clean speech signals positioned in different angles with respect to the sensors. With the training dataset, we use unsupervised learning to extract high-level features from low-level features and use supervised learning to find the nonlinear functions between high-level features and the orientations of dominant source. By using the trained networks, the probability that each T-F unit belongs to different sources (target and interferers) can be estimated based on the localization cues which is further used to generate the soft mask for source separation. Experiments based on real binaural RIRs and TIMIT dataset are provided to show the performance of the proposed system for reverberant speech mixtures, as compared with a model-based T-F masking technique proposed recently.

Bootstrap Averaging for Model-Based Source Separation in Reverberant Conditions

“…(18)]. Repeated iterations of the Eand M-steps are performed to obtain final estimates of the parameters, and subsequently ν iτ (ω, t) in the final E-step is computed using (19). Clearly, summing ν iτ (ω, t) over all possible delays τ gives the probability of the ith source being dominant at the time-frequency point (ω, t).…”

“…Note that the time-frequency mask given by (20) is a byproduct of the EM algorithm, computed using the output of the E-step (19) which gives the probability of a spectrogram point (ω, t) coming from source i and delay τ , conditional on the interaural cues α(ω, t) and ϕ(ω, t) (estimated using the spectrogram of the observed speech mixture), in addition to the parameter estimates from the final M-step of the EM algorithm. For each bootstrap replication, the E-step allows us to compute this probability or T-F masks, conditional on the interaural cues α * (ω, t) and ϕ * (ω, t), estimated from the spectrogram of the bootstrap speech mixture.…”

“…[17] and related work by [18], or using a probabilistic timefrequency mask, e.g. [19]. The latter approach integrates binaural cues following the model-based method of [8].…”

Binaural and log-power spectra features with deep neural networks for speech-noise separation

“…More recently, Deep Neural Networks (DNNs) [5] have shown the state-of-the-art performance in source separation [6]- [8].…”

“…The method proposed in [8], [9] extracts a target speech signal from a competing speech signal, using a DNN trained using binaural spatial cues of mixing vectors (MV), interaural level difference (ILD) and interaural phase difference (IPD). However, the above spatial cues become less effective for speech-noise scenarios where the target speech is often masked over by the background noise in adverse conditions.…”