This paper introduces a new method for multi-channel time domain speech separation in reverberant environments. A fullyconvolutional neural network structure has been used to directly separate speech from multiple microphone recordings, with no need of conventional spatial feature extraction. To reduce the influence of reverberation on spatial feature extraction, a dereverberation pre-processing method has been applied to further improve the separation performance. A spatialized version of wsj0-2mix dataset has been simulated to evaluate the proposed system. Both source separation and speech recognition performance of the separated signals have been evaluated objectively. Experiments show that the proposed fully-convolutional network improves the source separation metric and the word error rate (WER) by more than 13% and 50% relative, respectively, over a reference system with conventional features. Applying dereverberation as pre-processing to the proposed system can further reduce the WER by 29% relative using an acoustic model trained on clean and reverberated data.

This paper presents an automatic speech recognition (ASR) system for the 5th CHiME Speech Separation and Recognition Challenge for transcribing continuous conversations recorded in everyday environments with distributed microphone arrays. The main contribution of the proposed system is the investigation of an effective real-time channel selection scheme to pick up reliable microphones/array for target speakers. It is shown that the proposed channel selection method produces better ASR performance than the reference channel provided by the baseline system, as well as comparable results to a referencerequired selection approach based on speech intelligibility test as the oracle case. Instead of including all available data for training data augmentation, channel selection can be also applied to the training data selection to minimize the training-testmismatch. Further, complementary knowledge can be obtained when the best two channels are selected specially in periods of overlapping speech. The final ASR system, which additionally incorporates improved acoustic modeling and system combination, achieves absolute word error rate reductions of 9.2% and 7.0% with development and evaluation test set, respectively, compared to the baseline in the context of multiple-array track.

In this paper, we present a novel multi-channel speech extraction system to simultaneously extract multiple clean individual sources from a mixture in noisy and reverberant environments. The proposed method is built on an improved multi-channel time-domain speech separation network which employs speaker embeddings to identify and extract multiple targets without label permutation ambiguity. To efficiently inform the speaker information to the extraction model, we propose a new speaker conditioning mechanism by designing an additional speaker branch for receiving external speaker embeddings. Experiments on 2-channel WHAMR! data show that the proposed system improves by 9% relative the source separation performance over a strong multi-channel baseline, and it increases the speech recognition accuracy by more than 16% relative over the same baseline.

In this paper, we introduce a novel semi-supervised learning framework for end-to-end speech separation. The proposed method first uses mixtures of unseparated sources and the mixture invariant training (MixIT) criterion to train a teacher model. The teacher model then estimates separated sources that are used to train a student model with standard permutation invariant training (PIT). The student model can be fine-tuned with supervised data, i.e., paired artificial mixtures and clean speech sources, and further improved via model distillation. Experiments with single and multi channel mixtures show that the teacher-student training resolves the over-separation problem observed in the original MixIT method. Further, the semisupervised performance is comparable to a fully-supervised separation system trained using ten times the amount of supervised data.

No abstract